Script to append Timetamp in each line in case of missing time stamp












0














In GC logs, each line is having timestamp at the start. but sometimes we are missing that.



I am looking to write a bash script which check if Timestamp is missing it will add some static timestamp at the beginning.



In below example there are 2 lines. 1st line is having timestamp but 2nd line doesn't.



2018-11-22T11:58:39.381+0100: 79412.217: [GC (Allocation Failure) 2018-11-22T11:58:39.381+0100: 79412.217: [ParNew: 1265865K->36835K(1380160K), 0.1419160 secs] 23560553K->22332825K(29206784K), 0.1421816 secs] [Times: user=2.39 sys=0.00, real=0.14 secs] 
79412.947: [GC (Allocation Failure) 2018-11-22T11:58:40.112+0100: 79412.947: [ParNew: 1263651K->36868K(1380160K), 0.1502318 secs] 23559641K->22334516K(29206784K), 0.1504887 secs] [Times: user=2.47 sys=0.00, real=0.15 secs]



  1. Primary requirement is to add timestamp on each line if its missing.

  2. Advance: In case of no timestamp, there is one relative time(79412.947) in epoch format. I am looking to add some static epoch time on each relative time and append at the beginning.


For example In 2nd line: 79412.947 + 153434343 = some value. Convert this value is Timestamp format and append at the beginning.



Regex to extract timestamp.
https://regex101.com/r/3CtU4y/1/



Any suggestion, helpful advice is appreciated. Thank you in advance.










share|improve this question



























    0














    In GC logs, each line is having timestamp at the start. but sometimes we are missing that.



    I am looking to write a bash script which check if Timestamp is missing it will add some static timestamp at the beginning.



    In below example there are 2 lines. 1st line is having timestamp but 2nd line doesn't.



    2018-11-22T11:58:39.381+0100: 79412.217: [GC (Allocation Failure) 2018-11-22T11:58:39.381+0100: 79412.217: [ParNew: 1265865K->36835K(1380160K), 0.1419160 secs] 23560553K->22332825K(29206784K), 0.1421816 secs] [Times: user=2.39 sys=0.00, real=0.14 secs] 
    79412.947: [GC (Allocation Failure) 2018-11-22T11:58:40.112+0100: 79412.947: [ParNew: 1263651K->36868K(1380160K), 0.1502318 secs] 23559641K->22334516K(29206784K), 0.1504887 secs] [Times: user=2.47 sys=0.00, real=0.15 secs]



    1. Primary requirement is to add timestamp on each line if its missing.

    2. Advance: In case of no timestamp, there is one relative time(79412.947) in epoch format. I am looking to add some static epoch time on each relative time and append at the beginning.


    For example In 2nd line: 79412.947 + 153434343 = some value. Convert this value is Timestamp format and append at the beginning.



    Regex to extract timestamp.
    https://regex101.com/r/3CtU4y/1/



    Any suggestion, helpful advice is appreciated. Thank you in advance.










    share|improve this question

























      0












      0








      0







      In GC logs, each line is having timestamp at the start. but sometimes we are missing that.



      I am looking to write a bash script which check if Timestamp is missing it will add some static timestamp at the beginning.



      In below example there are 2 lines. 1st line is having timestamp but 2nd line doesn't.



      2018-11-22T11:58:39.381+0100: 79412.217: [GC (Allocation Failure) 2018-11-22T11:58:39.381+0100: 79412.217: [ParNew: 1265865K->36835K(1380160K), 0.1419160 secs] 23560553K->22332825K(29206784K), 0.1421816 secs] [Times: user=2.39 sys=0.00, real=0.14 secs] 
      79412.947: [GC (Allocation Failure) 2018-11-22T11:58:40.112+0100: 79412.947: [ParNew: 1263651K->36868K(1380160K), 0.1502318 secs] 23559641K->22334516K(29206784K), 0.1504887 secs] [Times: user=2.47 sys=0.00, real=0.15 secs]



      1. Primary requirement is to add timestamp on each line if its missing.

      2. Advance: In case of no timestamp, there is one relative time(79412.947) in epoch format. I am looking to add some static epoch time on each relative time and append at the beginning.


      For example In 2nd line: 79412.947 + 153434343 = some value. Convert this value is Timestamp format and append at the beginning.



      Regex to extract timestamp.
      https://regex101.com/r/3CtU4y/1/



      Any suggestion, helpful advice is appreciated. Thank you in advance.










      share|improve this question













      In GC logs, each line is having timestamp at the start. but sometimes we are missing that.



      I am looking to write a bash script which check if Timestamp is missing it will add some static timestamp at the beginning.



      In below example there are 2 lines. 1st line is having timestamp but 2nd line doesn't.



      2018-11-22T11:58:39.381+0100: 79412.217: [GC (Allocation Failure) 2018-11-22T11:58:39.381+0100: 79412.217: [ParNew: 1265865K->36835K(1380160K), 0.1419160 secs] 23560553K->22332825K(29206784K), 0.1421816 secs] [Times: user=2.39 sys=0.00, real=0.14 secs] 
      79412.947: [GC (Allocation Failure) 2018-11-22T11:58:40.112+0100: 79412.947: [ParNew: 1263651K->36868K(1380160K), 0.1502318 secs] 23559641K->22334516K(29206784K), 0.1504887 secs] [Times: user=2.47 sys=0.00, real=0.15 secs]



      1. Primary requirement is to add timestamp on each line if its missing.

      2. Advance: In case of no timestamp, there is one relative time(79412.947) in epoch format. I am looking to add some static epoch time on each relative time and append at the beginning.


      For example In 2nd line: 79412.947 + 153434343 = some value. Convert this value is Timestamp format and append at the beginning.



      Regex to extract timestamp.
      https://regex101.com/r/3CtU4y/1/



      Any suggestion, helpful advice is appreciated. Thank you in advance.







      bash timestamp






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 22 at 18:47









      Ankit Goyal

      669




      669
























          1 Answer
          1






          active

          oldest

          votes


















          1














          The first suggestion is that bash is probably not your best choice if you have a lot of logging.



          That aside, writing scripts is very much doing exactly what is asked. So it will be:



          1) set up a loop to read the input lines



          #!/bin/bash
          cat $* | while read line ; do


          2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.



          if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
          true


          Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.



          3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:



          else
          estamp=$(echo "$line" | sed 's/:.*//')
          rol=$(echo "$line" | sed 's/[0-9.]*://)
          # Do something with adding to the timestamp that I did not grasp
          echo "$estamp:$rol"
          fi


          Or something like that.



          O yeah, close the loop



          done





          share|improve this answer





















            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436675%2fscript-to-append-timetamp-in-each-line-in-case-of-missing-time-stamp%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            1














            The first suggestion is that bash is probably not your best choice if you have a lot of logging.



            That aside, writing scripts is very much doing exactly what is asked. So it will be:



            1) set up a loop to read the input lines



            #!/bin/bash
            cat $* | while read line ; do


            2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.



            if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
            true


            Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.



            3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:



            else
            estamp=$(echo "$line" | sed 's/:.*//')
            rol=$(echo "$line" | sed 's/[0-9.]*://)
            # Do something with adding to the timestamp that I did not grasp
            echo "$estamp:$rol"
            fi


            Or something like that.



            O yeah, close the loop



            done





            share|improve this answer


























              1














              The first suggestion is that bash is probably not your best choice if you have a lot of logging.



              That aside, writing scripts is very much doing exactly what is asked. So it will be:



              1) set up a loop to read the input lines



              #!/bin/bash
              cat $* | while read line ; do


              2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.



              if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
              true


              Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.



              3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:



              else
              estamp=$(echo "$line" | sed 's/:.*//')
              rol=$(echo "$line" | sed 's/[0-9.]*://)
              # Do something with adding to the timestamp that I did not grasp
              echo "$estamp:$rol"
              fi


              Or something like that.



              O yeah, close the loop



              done





              share|improve this answer
























                1












                1








                1






                The first suggestion is that bash is probably not your best choice if you have a lot of logging.



                That aside, writing scripts is very much doing exactly what is asked. So it will be:



                1) set up a loop to read the input lines



                #!/bin/bash
                cat $* | while read line ; do


                2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.



                if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
                true


                Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.



                3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:



                else
                estamp=$(echo "$line" | sed 's/:.*//')
                rol=$(echo "$line" | sed 's/[0-9.]*://)
                # Do something with adding to the timestamp that I did not grasp
                echo "$estamp:$rol"
                fi


                Or something like that.



                O yeah, close the loop



                done





                share|improve this answer












                The first suggestion is that bash is probably not your best choice if you have a lot of logging.



                That aside, writing scripts is very much doing exactly what is asked. So it will be:



                1) set up a loop to read the input lines



                #!/bin/bash
                cat $* | while read line ; do


                2) test if the input-line starts with a timestamp (did not bother looking up the regex you supplied.



                if echo "$line " | grep '^[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T[0-9][0-9]:[0-9][0-9]' ; then
                true


                Note that, if the condition is true, grep will output the line, so no separate echo is needed for that.



                3) Otherwise, do some things with the epoch timestamp that I can't quite understand what you want:



                else
                estamp=$(echo "$line" | sed 's/:.*//')
                rol=$(echo "$line" | sed 's/[0-9.]*://)
                # Do something with adding to the timestamp that I did not grasp
                echo "$estamp:$rol"
                fi


                Or something like that.



                O yeah, close the loop



                done






                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Nov 22 at 19:26









                Ljm Dullaart

                1,7761616




                1,7761616






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.





                    Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                    Please pay close attention to the following guidance:


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53436675%2fscript-to-append-timetamp-in-each-line-in-case-of-missing-time-stamp%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    What visual should I use to simply compare current year value vs last year in Power BI desktop

                    Alexandru Averescu

                    Trompette piccolo