Script to append Timetamp in each line in case of missing time stamp
In GC logs, each line is having timestamp at the start. but sometimes we are missing that.
I am looking to write a bash script which check if Timestamp is missing it will add some static timestamp at the beginning.
In below example there are 2 lines. 1st line is having timestamp but 2nd line doesn't.
2018-11-22T11:58:39.381+0100: 79412.217: [GC (Allocation Failure) 2018-11-22T11:58:39.381+0100: 79412.217: [ParNew: 1265865K->36835K(1380160K), 0.1419160 secs] 23560553K->22332825K(29206784K), 0.1421816 secs] [Times: user=2.39 sys=0.00, real=0.14 secs]
79412.947: [GC (Allocation Failure) 2018-11-22T11:58:40.112+0100: 79412.947: [ParNew: 1263651K->36868K(1380160K), 0.1502318 secs] 23559641K->22334516K(29206784K), 0.1504887 secs] [Times: user=2.47 sys=0.00, real=0.15 secs]
- Primary requirement is to add timestamp on each line if its missing.
- Advance: In case of no timestamp, there is one relative time(79412.947) in epoch format. I am looking to add some static epoch time on each relative time and append at the beginning.
For example In 2nd line: 79412.947 + 153434343 = some value. Convert this value is Timestamp format and append at the beginning.
Regex to extract timestamp.
https://regex101.com/r/3CtU4y/1/
Any suggestion, helpful advice is appreciated. Thank you in advance.
bash timestamp
asked Nov 22 at 18:47
Ankit Goyal
669
add a comment |
In GC logs, each line is having timestamp at the start. but sometimes we are missing that.
I am looking to write a bash script which check if Timestamp is missing it will add some static timestamp at the beginning.
In below example there are 2 lines. 1st line is having timestamp but 2nd line doesn't.
2018-11-22T11:58:39.381+0100: 79412.217: [GC (Allocation Failure) 2018-11-22T11:58:39.381+0100: 79412.217: [ParNew: 1265865K->36835K(1380160K), 0.1419160 secs] 23560553K->22332825K(29206784K), 0.1421816 secs] [Times: user=2.39 sys=0.00, real=0.14 secs]
79412.947: [GC (Allocation Failure) 2018-11-22T11:58:40.112+0100: 79412.947: [ParNew: 1263651K->36868K(1380160K), 0.1502318 secs] 23559641K->22334516K(29206784K), 0.1504887 secs] [Times: user=2.47 sys=0.00, real=0.15 secs]
- Primary requirement is to add timestamp on each line if its missing.
- Advance: In case of no timestamp, there is one relative time(79412.947) in epoch format. I am looking to add some static epoch time on each relative time and append at the beginning.
For example In 2nd line: 79412.947 + 153434343 = some value. Convert this value is Timestamp format and append at the beginning.
Regex to extract timestamp.
https://regex101.com/r/3CtU4y/1/
Any suggestion, helpful advice is appreciated. Thank you in advance.
bash timestamp
asked Nov 22 at 18:47
Ankit Goyal
669
add a comment |
In GC logs, each line is having timestamp at the start. but sometimes we are missing that.
I am looking to write a bash script which check if Timestamp is missing it will add some static timestamp at the beginning.
In below example there are 2 lines. 1st line is having timestamp but 2nd line doesn't.
2018-11-22T11:58:39.381+0100: 79412.217: [GC (Allocation Failure) 2018-11-22T11:58:39.381+0100: 79412.217: [ParNew: 1265865K->36835K(1380160K), 0.1419160 secs] 23560553K->22332825K(29206784K), 0.1421816 secs] [Times: user=2.39 sys=0.00, real=0.14 secs]
79412.947: [GC (Allocation Failure) 2018-11-22T11:58:40.112+0100: 79412.947: [ParNew: 1263651K->36868K(1380160K), 0.1502318 secs] 23559641K->22334516K(29206784K), 0.1504887 secs] [Times: user=2.47 sys=0.00, real=0.15 secs]
- Primary requirement is to add timestamp on each line if its missing.
- Advance: In case of no timestamp, there is one relative time(79412.947) in epoch format. I am looking to add some static epoch time on each relative time and append at the beginning.
For example In 2nd line: 79412.947 + 153434343 = some value. Convert this value is Timestamp format and append at the beginning.
Regex to extract timestamp.
https://regex101.com/r/3CtU4y/1/
Any suggestion, helpful advice is appreciated. Thank you in advance.
bash timestamp
asked Nov 22 at 18:47
Ankit Goyal
669
In GC logs, each line is having timestamp at the start. but sometimes we are missing that.
I am looking to write a bash script which check if Timestamp is missing it will add some static timestamp at the beginning.
In below example there are 2 lines. 1st line is having timestamp but 2nd line doesn't.
2018-11-22T11:58:39.381+0100: 79412.217: [GC (Allocation Failure) 2018-11-22T11:58:39.381+0100: 79412.217: [ParNew: 1265865K->36835K(1380160K), 0.1419160 secs] 23560553K->22332825K(29206784K), 0.1421816 secs] [Times: user=2.39 sys=0.00, real=0.14 secs]
79412.947: [GC (Allocation Failure) 2018-11-22T11:58:40.112+0100: 79412.947: [ParNew: 1263651K->36868K(1380160K), 0.1502318 secs] 23559641K->22334516K(29206784K), 0.1504887 secs] [Times: user=2.47 sys=0.00, real=0.15 secs]
- Primary requirement is to add timestamp on each line if its missing.
- Advance: In case of no timestamp, there is one relative time(79412.947) in epoch format. I am looking to add some static epoch time on each relative time and append at the beginning.
For example In 2nd line: 79412.947 + 153434343 = some value. Convert this value is Timestamp format and append at the beginning.
Regex to extract timestamp.
https://regex101.com/r/3CtU4y/1/
Any suggestion, helpful advice is appreciated. Thank you in advance.
bash timestamp
bash timestamp
asked Nov 22 at 18:47
Ankit Goyal
669
asked Nov 22 at 18:47
Ankit Goyal
669
asked Nov 22 at 18:47
Ankit Goyal
669
asked Nov 22 at 18:47
Ankit Goyal
669
asked Nov 22 at 18:47
Ankit Goyal
669
669
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
The first suggestion is that bash is probably not your best choice if you have a lot of logging.
That aside, writing scripts is very much doing exactly what is asked. So it will be:
1) set up a loop to read the input lines
#!/bin/bash
cat $* | while read line ; do
2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.
if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
true
Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.
3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:
else
estamp=$(echo "$line" | sed 's/:.*//')
rol=$(echo "$line" | sed 's/[0-9.]*://)
# Do something with adding to the timestamp that I did not grasp
echo "$estamp:$rol"
fi
Or something like that.
O yeah, close the loop
done
answered Nov 22 at 19:26
Ljm Dullaart
1,7761616
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436675%2fscript-to-append-timetamp-in-each-line-in-case-of-missing-time-stamp%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The first suggestion is that bash is probably not your best choice if you have a lot of logging.
That aside, writing scripts is very much doing exactly what is asked. So it will be:
1) set up a loop to read the input lines
#!/bin/bash
cat $* | while read line ; do
2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.
if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
true
Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.
3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:
else
estamp=$(echo "$line" | sed 's/:.*//')
rol=$(echo "$line" | sed 's/[0-9.]*://)
# Do something with adding to the timestamp that I did not grasp
echo "$estamp:$rol"
fi
Or something like that.
O yeah, close the loop
done
answered Nov 22 at 19:26
Ljm Dullaart
1,7761616
add a comment |
The first suggestion is that bash is probably not your best choice if you have a lot of logging.
That aside, writing scripts is very much doing exactly what is asked. So it will be:
1) set up a loop to read the input lines
#!/bin/bash
cat $* | while read line ; do
2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.
if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
true
Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.
3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:
else
estamp=$(echo "$line" | sed 's/:.*//')
rol=$(echo "$line" | sed 's/[0-9.]*://)
# Do something with adding to the timestamp that I did not grasp
echo "$estamp:$rol"
fi
Or something like that.
O yeah, close the loop
done
answered Nov 22 at 19:26
Ljm Dullaart
1,7761616
add a comment |
The first suggestion is that bash is probably not your best choice if you have a lot of logging.
That aside, writing scripts is very much doing exactly what is asked. So it will be:
1) set up a loop to read the input lines
#!/bin/bash
cat $* | while read line ; do
2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.
if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
true
Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.
3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:
else
estamp=$(echo "$line" | sed 's/:.*//')
rol=$(echo "$line" | sed 's/[0-9.]*://)
# Do something with adding to the timestamp that I did not grasp
echo "$estamp:$rol"
fi
Or something like that.
O yeah, close the loop
done
answered Nov 22 at 19:26
Ljm Dullaart
1,7761616
The first suggestion is that bash is probably not your best choice if you have a lot of logging.
That aside, writing scripts is very much doing exactly what is asked. So it will be:
1) set up a loop to read the input lines
#!/bin/bash
cat $* | while read line ; do
2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.
if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
true
Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.
3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:
else
estamp=$(echo "$line" | sed 's/:.*//')
rol=$(echo "$line" | sed 's/[0-9.]*://)
# Do something with adding to the timestamp that I did not grasp
echo "$estamp:$rol"
fi
Or something like that.
O yeah, close the loop
done
answered Nov 22 at 19:26
Ljm Dullaart
1,7761616
answered Nov 22 at 19:26
Ljm Dullaart
1,7761616
answered Nov 22 at 19:26
Ljm Dullaart
1,7761616
answered Nov 22 at 19:26
Ljm Dullaart
1,7761616
1,7761616
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436675%2fscript-to-append-timetamp-in-each-line-in-case-of-missing-time-stamp%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
