How do I delete lines of 1st file if it matches the string present in the 2nd file in linux?
Consider I have two text files.
First File name - "Emails.txt" with the following data:
00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00eeeeee@gotmail.com
00gggggg@uor.edu
00uuuuuuuu@yahoo.com
00e21_ss@cmail.com
00gggggggg@cmail.com
00zzzzzzzz48@hotmail.com
00aaaaaaa_2020@gotmail.com
jjjjjjjj@gmail.com
Second text file - "Banned.txt" with the following strings:
@gotmail.com
@cmail.com
@uor.edu
How to delete all the lines in the 1st text file "Emails.txt" if it matches the stings of any line present in the second text file "Banned.txt"?
The desired output of the new file should be:
00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00uuuuuuuu@yahoo.com
00zzzzzzzz48@hotmail.com
jjjjjjjj@gmail.com
Can this be done using SED or awk in Linux? Can you please suggest how to do this?
linux command-line sed
edited Jan 23 at 12:22
grawity
240k37508562
asked Jan 23 at 12:01
Joney WalkerJoney Walker
235
add a comment |
Consider I have two text files.
First File name - "Emails.txt" with the following data:
00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00eeeeee@gotmail.com
00gggggg@uor.edu
00uuuuuuuu@yahoo.com
00e21_ss@cmail.com
00gggggggg@cmail.com
00zzzzzzzz48@hotmail.com
00aaaaaaa_2020@gotmail.com
jjjjjjjj@gmail.com
Second text file - "Banned.txt" with the following strings:
@gotmail.com
@cmail.com
@uor.edu
How to delete all the lines in the 1st text file "Emails.txt" if it matches the stings of any line present in the second text file "Banned.txt"?
The desired output of the new file should be:
00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00uuuuuuuu@yahoo.com
00zzzzzzzz48@hotmail.com
jjjjjjjj@gmail.com
Can this be done using SED or awk in Linux? Can you please suggest how to do this?
linux command-line sed
edited Jan 23 at 12:22
grawity
240k37508562
asked Jan 23 at 12:01
Joney WalkerJoney Walker
235
add a comment |
Consider I have two text files.
First File name - "Emails.txt" with the following data:
00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00eeeeee@gotmail.com
00gggggg@uor.edu
00uuuuuuuu@yahoo.com
00e21_ss@cmail.com
00gggggggg@cmail.com
00zzzzzzzz48@hotmail.com
00aaaaaaa_2020@gotmail.com
jjjjjjjj@gmail.com
Second text file - "Banned.txt" with the following strings:
@gotmail.com
@cmail.com
@uor.edu
How to delete all the lines in the 1st text file "Emails.txt" if it matches the stings of any line present in the second text file "Banned.txt"?
The desired output of the new file should be:
00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00uuuuuuuu@yahoo.com
00zzzzzzzz48@hotmail.com
jjjjjjjj@gmail.com
Can this be done using SED or awk in Linux? Can you please suggest how to do this?
linux command-line sed
edited Jan 23 at 12:22
grawity
240k37508562
asked Jan 23 at 12:01
Joney WalkerJoney Walker
235
Consider I have two text files.
First File name - "Emails.txt" with the following data:
00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00eeeeee@gotmail.com
00gggggg@uor.edu
00uuuuuuuu@yahoo.com
00e21_ss@cmail.com
00gggggggg@cmail.com
00zzzzzzzz48@hotmail.com
00aaaaaaa_2020@gotmail.com
jjjjjjjj@gmail.com
Second text file - "Banned.txt" with the following strings:
@gotmail.com
@cmail.com
@uor.edu
How to delete all the lines in the 1st text file "Emails.txt" if it matches the stings of any line present in the second text file "Banned.txt"?
The desired output of the new file should be:
00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00uuuuuuuu@yahoo.com
00zzzzzzzz48@hotmail.com
jjjjjjjj@gmail.com
Can this be done using SED or awk in Linux? Can you please suggest how to do this?
linux command-line sed
linux command-line sed
edited Jan 23 at 12:22
grawity
240k37508562
asked Jan 23 at 12:01
Joney WalkerJoney Walker
235
edited Jan 23 at 12:22
grawity
240k37508562
asked Jan 23 at 12:01
Joney WalkerJoney Walker
235
edited Jan 23 at 12:22
grawity
240k37508562
edited Jan 23 at 12:22
grawity
240k37508562
edited Jan 23 at 12:22
grawity
240k37508562
240k37508562
asked Jan 23 at 12:01
Joney WalkerJoney Walker
235
asked Jan 23 at 12:01
Joney WalkerJoney Walker
235
asked Jan 23 at 12:01
Joney WalkerJoney Walker
235
235
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
grep -v is enough. The flag -f allows you to do exactly what you want:
grep -vf Banned.txt Emails.txt
If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'
gives the desired
@gotmail.com$|@cmail.com$|@uor.edu$
Then:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt
(doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.
answered Jan 23 at 12:34
JoceJoce
5001318
This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt
– Joney Walker
Jan 23 at 12:46
That's the purpose of the-v. Have you tried it?
– Joce
Jan 23 at 12:48
1
grep -vf worked like a charm... Thanks.
– Joney Walker
Jan 23 at 12:58
1
After some more research, I found - Rather using a long regex, this works with this simple line too:grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt
– Joney Walker
Jan 28 at 12:04
1
@JoneyWalker Indeed that's shorter!
– Joce
Jan 28 at 15:27
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1397436%2fhow-do-i-delete-lines-of-1st-file-if-it-matches-the-string-present-in-the-2nd-fi%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
grep -v is enough. The flag -f allows you to do exactly what you want:
grep -vf Banned.txt Emails.txt
If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'
gives the desired
@gotmail.com$|@cmail.com$|@uor.edu$
Then:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt
(doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.
answered Jan 23 at 12:34
JoceJoce
5001318
This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt
– Joney Walker
Jan 23 at 12:46
That's the purpose of the-v. Have you tried it?
– Joce
Jan 23 at 12:48
1
grep -vf worked like a charm... Thanks.
– Joney Walker
Jan 23 at 12:58
1
After some more research, I found - Rather using a long regex, this works with this simple line too:grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt
– Joney Walker
Jan 28 at 12:04
1
@JoneyWalker Indeed that's shorter!
– Joce
Jan 28 at 15:27
add a comment |
grep -v is enough. The flag -f allows you to do exactly what you want:
grep -vf Banned.txt Emails.txt
If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'
gives the desired
@gotmail.com$|@cmail.com$|@uor.edu$
Then:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt
(doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.
answered Jan 23 at 12:34
JoceJoce
5001318
This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt
– Joney Walker
Jan 23 at 12:46
That's the purpose of the-v. Have you tried it?
– Joce
Jan 23 at 12:48
1
grep -vf worked like a charm... Thanks.
– Joney Walker
Jan 23 at 12:58
1
After some more research, I found - Rather using a long regex, this works with this simple line too:grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt
– Joney Walker
Jan 28 at 12:04
1
@JoneyWalker Indeed that's shorter!
– Joce
Jan 28 at 15:27
add a comment |
grep -v is enough. The flag -f allows you to do exactly what you want:
grep -vf Banned.txt Emails.txt
If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'
gives the desired
@gotmail.com$|@cmail.com$|@uor.edu$
Then:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt
(doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.
answered Jan 23 at 12:34
JoceJoce
5001318
grep -v is enough. The flag -f allows you to do exactly what you want:
grep -vf Banned.txt Emails.txt
If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'
gives the desired
@gotmail.com$|@cmail.com$|@uor.edu$
Then:
cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt
(doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.
answered Jan 23 at 12:34
JoceJoce
5001318
edited Jan 24 at 14:15
answered Jan 23 at 12:34
JoceJoce
5001318
answered Jan 23 at 12:34
JoceJoce
5001318
answered Jan 23 at 12:34
JoceJoce
5001318
5001318
This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt
– Joney Walker
Jan 23 at 12:46
That's the purpose of the-v. Have you tried it?
– Joce
Jan 23 at 12:48
1
grep -vf worked like a charm... Thanks.
– Joney Walker
Jan 23 at 12:58
1
After some more research, I found - Rather using a long regex, this works with this simple line too:grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt
– Joney Walker
Jan 28 at 12:04
1
@JoneyWalker Indeed that's shorter!
– Joce
Jan 28 at 15:27
add a comment |
This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt
– Joney Walker
Jan 23 at 12:46
That's the purpose of the-v. Have you tried it?
– Joce
Jan 23 at 12:48
1
grep -vf worked like a charm... Thanks.
– Joney Walker
Jan 23 at 12:58
1
After some more research, I found - Rather using a long regex, this works with this simple line too:grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt
– Joney Walker
Jan 28 at 12:04
1
@JoneyWalker Indeed that's shorter!
– Joce
Jan 28 at 15:27
This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt
– Joney Walker
Jan 23 at 12:46
This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt
– Joney Walker
Jan 23 at 12:46
That's the purpose of the
-v. Have you tried it?– Joce
Jan 23 at 12:48
That's the purpose of the
-v. Have you tried it?– Joce
Jan 23 at 12:48
1
1
grep -vf worked like a charm... Thanks.
– Joney Walker
Jan 23 at 12:58
grep -vf worked like a charm... Thanks.
– Joney Walker
Jan 23 at 12:58
1
1
After some more research, I found - Rather using a long regex, this works with this simple line too:
grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt– Joney Walker
Jan 28 at 12:04
After some more research, I found - Rather using a long regex, this works with this simple line too:
grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt– Joney Walker
Jan 28 at 12:04
1
1
@JoneyWalker Indeed that's shorter!
– Joce
Jan 28 at 15:27
@JoneyWalker Indeed that's shorter!
– Joce
Jan 28 at 15:27
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1397436%2fhow-do-i-delete-lines-of-1st-file-if-it-matches-the-string-present-in-the-2nd-fi%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
