How does the asyncio module work, why is my updated sample running synchronously?











up vote
2
down vote

favorite
1












I have tried the following code in Python 3.6 for asyncio:
Example 1:



import asyncio
import time

async def hello():

print('hello')
await asyncio.sleep(1)
print('hello again')

tasks=[hello(),hello()]
loop=asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(tasks))


Output is as expected:



hello
hello
hello again
hello again


Then I want to change the asyncio.sleep into another def:



async def sleep():
time.sleep(1)

async def hello():

print('hello')
await sleep()
print('hello again')


tasks=[hello(),hello()]
loop=asyncio.get_event_loop()
loop.run_until_complete(asyncio.wait(tasks))


Output:



hello
hello again
hello
hello again


It seems it is not running in an asynchronous mode, but a normal sync mode.



The question is: Why is it not running in an asynchronous mode and how can I change the old sync module into an 'async' one?










share|improve this question




























    up vote
    2
    down vote

    favorite
    1












    I have tried the following code in Python 3.6 for asyncio:
    Example 1:



    import asyncio
    import time

    async def hello():

    print('hello')
    await asyncio.sleep(1)
    print('hello again')

    tasks=[hello(),hello()]
    loop=asyncio.get_event_loop()
    loop.run_until_complete(asyncio.wait(tasks))


    Output is as expected:



    hello
    hello
    hello again
    hello again


    Then I want to change the asyncio.sleep into another def:



    async def sleep():
    time.sleep(1)

    async def hello():

    print('hello')
    await sleep()
    print('hello again')


    tasks=[hello(),hello()]
    loop=asyncio.get_event_loop()
    loop.run_until_complete(asyncio.wait(tasks))


    Output:



    hello
    hello again
    hello
    hello again


    It seems it is not running in an asynchronous mode, but a normal sync mode.



    The question is: Why is it not running in an asynchronous mode and how can I change the old sync module into an 'async' one?










    share|improve this question


























      up vote
      2
      down vote

      favorite
      1









      up vote
      2
      down vote

      favorite
      1






      1





      I have tried the following code in Python 3.6 for asyncio:
      Example 1:



      import asyncio
      import time

      async def hello():

      print('hello')
      await asyncio.sleep(1)
      print('hello again')

      tasks=[hello(),hello()]
      loop=asyncio.get_event_loop()
      loop.run_until_complete(asyncio.wait(tasks))


      Output is as expected:



      hello
      hello
      hello again
      hello again


      Then I want to change the asyncio.sleep into another def:



      async def sleep():
      time.sleep(1)

      async def hello():

      print('hello')
      await sleep()
      print('hello again')


      tasks=[hello(),hello()]
      loop=asyncio.get_event_loop()
      loop.run_until_complete(asyncio.wait(tasks))


      Output:



      hello
      hello again
      hello
      hello again


      It seems it is not running in an asynchronous mode, but a normal sync mode.



      The question is: Why is it not running in an asynchronous mode and how can I change the old sync module into an 'async' one?










      share|improve this question















      I have tried the following code in Python 3.6 for asyncio:
      Example 1:



      import asyncio
      import time

      async def hello():

      print('hello')
      await asyncio.sleep(1)
      print('hello again')

      tasks=[hello(),hello()]
      loop=asyncio.get_event_loop()
      loop.run_until_complete(asyncio.wait(tasks))


      Output is as expected:



      hello
      hello
      hello again
      hello again


      Then I want to change the asyncio.sleep into another def:



      async def sleep():
      time.sleep(1)

      async def hello():

      print('hello')
      await sleep()
      print('hello again')


      tasks=[hello(),hello()]
      loop=asyncio.get_event_loop()
      loop.run_until_complete(asyncio.wait(tasks))


      Output:



      hello
      hello again
      hello
      hello again


      It seems it is not running in an asynchronous mode, but a normal sync mode.



      The question is: Why is it not running in an asynchronous mode and how can I change the old sync module into an 'async' one?







      python async-await python-asyncio






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Jul 26 '17 at 7:32









      Martijn Pieters

      694k12924032244




      694k12924032244










      asked Jul 26 '17 at 7:11









      user7936694

      183




      183
























          1 Answer
          1






          active

          oldest

          votes

















          up vote
          8
          down vote



          accepted










          Asyncio uses an event loop, which selects what task (an independent call chain of coroutines) in the queue to activate next. The event loop can make intelligent decisions as to what task is ready to do actual work. This is why the event loop also is responsible for creating connections and watching file descriptors and other I/O primitives; it gives the event loop insight into when there are I/O operations in progress or when results are available to process.



          Whenever you use await, there is an opportunity to return control to the loop which can then pass control to another task. Which task then is picked for execution depends on the exact implementation; the asyncio reference implementation offers multiple choices, but there are other implementations, such as the very, very efficient uvloop implementation.



          Your sample is still asynchronous. It just so happens that by replacing the await.sleep() with a synchronous time.sleep() call, inside a new coroutine function, you introduced 2 coroutines into the task callchain that don't yield, and thus influenced in what order they are executed. That they are executed in what appears to be synchronous order is a coincidence. If you switched event loops, or introduced more coroutines (especially some that use I/O), the order can easily be different again.



          Moreover, your new coroutines use time.sleep(); this makes your coroutines uncooperative. The event loop is not notified that your code is waiting (time.sleep() will not yield!), so no other coroutine can be executed while time.sleep() is running. time.sleep() simply doesn't return or lets any other code run until the requested amount of time has passed. Contrast this with the asyncio.sleep() implementation, which simply yields to the event loop with a call_later() hook; the event loop now knows that that task won't need any attention until a later time.



          Also see asyncio: why isn't it non-blocking by default for a more in-depth discussion of how tasks and the event loop interact. And if you must run blocking, synchronous code that can't be made to cooperate, then use an executor pool to have the blocking code executed in a separate tread or child process to free up the event loop for other, better behaved tasks.






          share|improve this answer























          • Hi Martijn, Thanks a lot for the answer. then if I have an old sync def or method, how can I change them into an asynchronous one?
            – user7936694
            Jul 26 '17 at 13:22










          • You must either rewrite the function in async manner (there are many ready libs e.g. for pg, redis, http... mainly some io stuff) or use executors (mainly for use with cpu-intensive stuff) like ThreadPoolExecutro and ProcessPoolExecutor (docs.python.org/3/library/…). More examples can be found at pymotw.com/3/asyncio/executors.html
            – kwarunek
            Jul 26 '17 at 14:39












          • @user7936694: exactly what kwarunek said; put code you can't make async in an executor, which then pretends your code is async by executing it under a different thread or in a separate process.
            – Martijn Pieters
            Jul 26 '17 at 14:48










          • Hi Martijn, thank a lot, seems it will take a while to change my old code into an asynchronous one :)
            – user7936694
            Jul 26 '17 at 19:01











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f45319890%2fhow-does-the-asyncio-module-work-why-is-my-updated-sample-running-synchronously%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          8
          down vote



          accepted










          Asyncio uses an event loop, which selects what task (an independent call chain of coroutines) in the queue to activate next. The event loop can make intelligent decisions as to what task is ready to do actual work. This is why the event loop also is responsible for creating connections and watching file descriptors and other I/O primitives; it gives the event loop insight into when there are I/O operations in progress or when results are available to process.



          Whenever you use await, there is an opportunity to return control to the loop which can then pass control to another task. Which task then is picked for execution depends on the exact implementation; the asyncio reference implementation offers multiple choices, but there are other implementations, such as the very, very efficient uvloop implementation.



          Your sample is still asynchronous. It just so happens that by replacing the await.sleep() with a synchronous time.sleep() call, inside a new coroutine function, you introduced 2 coroutines into the task callchain that don't yield, and thus influenced in what order they are executed. That they are executed in what appears to be synchronous order is a coincidence. If you switched event loops, or introduced more coroutines (especially some that use I/O), the order can easily be different again.



          Moreover, your new coroutines use time.sleep(); this makes your coroutines uncooperative. The event loop is not notified that your code is waiting (time.sleep() will not yield!), so no other coroutine can be executed while time.sleep() is running. time.sleep() simply doesn't return or lets any other code run until the requested amount of time has passed. Contrast this with the asyncio.sleep() implementation, which simply yields to the event loop with a call_later() hook; the event loop now knows that that task won't need any attention until a later time.



          Also see asyncio: why isn't it non-blocking by default for a more in-depth discussion of how tasks and the event loop interact. And if you must run blocking, synchronous code that can't be made to cooperate, then use an executor pool to have the blocking code executed in a separate tread or child process to free up the event loop for other, better behaved tasks.






          share|improve this answer























          • Hi Martijn, Thanks a lot for the answer. then if I have an old sync def or method, how can I change them into an asynchronous one?
            – user7936694
            Jul 26 '17 at 13:22










          • You must either rewrite the function in async manner (there are many ready libs e.g. for pg, redis, http... mainly some io stuff) or use executors (mainly for use with cpu-intensive stuff) like ThreadPoolExecutro and ProcessPoolExecutor (docs.python.org/3/library/…). More examples can be found at pymotw.com/3/asyncio/executors.html
            – kwarunek
            Jul 26 '17 at 14:39












          • @user7936694: exactly what kwarunek said; put code you can't make async in an executor, which then pretends your code is async by executing it under a different thread or in a separate process.
            – Martijn Pieters
            Jul 26 '17 at 14:48










          • Hi Martijn, thank a lot, seems it will take a while to change my old code into an asynchronous one :)
            – user7936694
            Jul 26 '17 at 19:01















          up vote
          8
          down vote



          accepted










          Asyncio uses an event loop, which selects what task (an independent call chain of coroutines) in the queue to activate next. The event loop can make intelligent decisions as to what task is ready to do actual work. This is why the event loop also is responsible for creating connections and watching file descriptors and other I/O primitives; it gives the event loop insight into when there are I/O operations in progress or when results are available to process.



          Whenever you use await, there is an opportunity to return control to the loop which can then pass control to another task. Which task then is picked for execution depends on the exact implementation; the asyncio reference implementation offers multiple choices, but there are other implementations, such as the very, very efficient uvloop implementation.



          Your sample is still asynchronous. It just so happens that by replacing the await.sleep() with a synchronous time.sleep() call, inside a new coroutine function, you introduced 2 coroutines into the task callchain that don't yield, and thus influenced in what order they are executed. That they are executed in what appears to be synchronous order is a coincidence. If you switched event loops, or introduced more coroutines (especially some that use I/O), the order can easily be different again.



          Moreover, your new coroutines use time.sleep(); this makes your coroutines uncooperative. The event loop is not notified that your code is waiting (time.sleep() will not yield!), so no other coroutine can be executed while time.sleep() is running. time.sleep() simply doesn't return or lets any other code run until the requested amount of time has passed. Contrast this with the asyncio.sleep() implementation, which simply yields to the event loop with a call_later() hook; the event loop now knows that that task won't need any attention until a later time.



          Also see asyncio: why isn't it non-blocking by default for a more in-depth discussion of how tasks and the event loop interact. And if you must run blocking, synchronous code that can't be made to cooperate, then use an executor pool to have the blocking code executed in a separate tread or child process to free up the event loop for other, better behaved tasks.






          share|improve this answer























          • Hi Martijn, Thanks a lot for the answer. then if I have an old sync def or method, how can I change them into an asynchronous one?
            – user7936694
            Jul 26 '17 at 13:22










          • You must either rewrite the function in async manner (there are many ready libs e.g. for pg, redis, http... mainly some io stuff) or use executors (mainly for use with cpu-intensive stuff) like ThreadPoolExecutro and ProcessPoolExecutor (docs.python.org/3/library/…). More examples can be found at pymotw.com/3/asyncio/executors.html
            – kwarunek
            Jul 26 '17 at 14:39












          • @user7936694: exactly what kwarunek said; put code you can't make async in an executor, which then pretends your code is async by executing it under a different thread or in a separate process.
            – Martijn Pieters
            Jul 26 '17 at 14:48










          • Hi Martijn, thank a lot, seems it will take a while to change my old code into an asynchronous one :)
            – user7936694
            Jul 26 '17 at 19:01













          up vote
          8
          down vote



          accepted







          up vote
          8
          down vote



          accepted






          Asyncio uses an event loop, which selects what task (an independent call chain of coroutines) in the queue to activate next. The event loop can make intelligent decisions as to what task is ready to do actual work. This is why the event loop also is responsible for creating connections and watching file descriptors and other I/O primitives; it gives the event loop insight into when there are I/O operations in progress or when results are available to process.



          Whenever you use await, there is an opportunity to return control to the loop which can then pass control to another task. Which task then is picked for execution depends on the exact implementation; the asyncio reference implementation offers multiple choices, but there are other implementations, such as the very, very efficient uvloop implementation.



          Your sample is still asynchronous. It just so happens that by replacing the await.sleep() with a synchronous time.sleep() call, inside a new coroutine function, you introduced 2 coroutines into the task callchain that don't yield, and thus influenced in what order they are executed. That they are executed in what appears to be synchronous order is a coincidence. If you switched event loops, or introduced more coroutines (especially some that use I/O), the order can easily be different again.



          Moreover, your new coroutines use time.sleep(); this makes your coroutines uncooperative. The event loop is not notified that your code is waiting (time.sleep() will not yield!), so no other coroutine can be executed while time.sleep() is running. time.sleep() simply doesn't return or lets any other code run until the requested amount of time has passed. Contrast this with the asyncio.sleep() implementation, which simply yields to the event loop with a call_later() hook; the event loop now knows that that task won't need any attention until a later time.



          Also see asyncio: why isn't it non-blocking by default for a more in-depth discussion of how tasks and the event loop interact. And if you must run blocking, synchronous code that can't be made to cooperate, then use an executor pool to have the blocking code executed in a separate tread or child process to free up the event loop for other, better behaved tasks.






          share|improve this answer














          Asyncio uses an event loop, which selects what task (an independent call chain of coroutines) in the queue to activate next. The event loop can make intelligent decisions as to what task is ready to do actual work. This is why the event loop also is responsible for creating connections and watching file descriptors and other I/O primitives; it gives the event loop insight into when there are I/O operations in progress or when results are available to process.



          Whenever you use await, there is an opportunity to return control to the loop which can then pass control to another task. Which task then is picked for execution depends on the exact implementation; the asyncio reference implementation offers multiple choices, but there are other implementations, such as the very, very efficient uvloop implementation.



          Your sample is still asynchronous. It just so happens that by replacing the await.sleep() with a synchronous time.sleep() call, inside a new coroutine function, you introduced 2 coroutines into the task callchain that don't yield, and thus influenced in what order they are executed. That they are executed in what appears to be synchronous order is a coincidence. If you switched event loops, or introduced more coroutines (especially some that use I/O), the order can easily be different again.



          Moreover, your new coroutines use time.sleep(); this makes your coroutines uncooperative. The event loop is not notified that your code is waiting (time.sleep() will not yield!), so no other coroutine can be executed while time.sleep() is running. time.sleep() simply doesn't return or lets any other code run until the requested amount of time has passed. Contrast this with the asyncio.sleep() implementation, which simply yields to the event loop with a call_later() hook; the event loop now knows that that task won't need any attention until a later time.



          Also see asyncio: why isn't it non-blocking by default for a more in-depth discussion of how tasks and the event loop interact. And if you must run blocking, synchronous code that can't be made to cooperate, then use an executor pool to have the blocking code executed in a separate tread or child process to free up the event loop for other, better behaved tasks.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Nov 22 at 12:05

























          answered Jul 26 '17 at 7:30









          Martijn Pieters

          694k12924032244




          694k12924032244












          • Hi Martijn, Thanks a lot for the answer. then if I have an old sync def or method, how can I change them into an asynchronous one?
            – user7936694
            Jul 26 '17 at 13:22










          • You must either rewrite the function in async manner (there are many ready libs e.g. for pg, redis, http... mainly some io stuff) or use executors (mainly for use with cpu-intensive stuff) like ThreadPoolExecutro and ProcessPoolExecutor (docs.python.org/3/library/…). More examples can be found at pymotw.com/3/asyncio/executors.html
            – kwarunek
            Jul 26 '17 at 14:39












          • @user7936694: exactly what kwarunek said; put code you can't make async in an executor, which then pretends your code is async by executing it under a different thread or in a separate process.
            – Martijn Pieters
            Jul 26 '17 at 14:48










          • Hi Martijn, thank a lot, seems it will take a while to change my old code into an asynchronous one :)
            – user7936694
            Jul 26 '17 at 19:01


















          • Hi Martijn, Thanks a lot for the answer. then if I have an old sync def or method, how can I change them into an asynchronous one?
            – user7936694
            Jul 26 '17 at 13:22










          • You must either rewrite the function in async manner (there are many ready libs e.g. for pg, redis, http... mainly some io stuff) or use executors (mainly for use with cpu-intensive stuff) like ThreadPoolExecutro and ProcessPoolExecutor (docs.python.org/3/library/…). More examples can be found at pymotw.com/3/asyncio/executors.html
            – kwarunek
            Jul 26 '17 at 14:39












          • @user7936694: exactly what kwarunek said; put code you can't make async in an executor, which then pretends your code is async by executing it under a different thread or in a separate process.
            – Martijn Pieters
            Jul 26 '17 at 14:48










          • Hi Martijn, thank a lot, seems it will take a while to change my old code into an asynchronous one :)
            – user7936694
            Jul 26 '17 at 19:01
















          Hi Martijn, Thanks a lot for the answer. then if I have an old sync def or method, how can I change them into an asynchronous one?
          – user7936694
          Jul 26 '17 at 13:22




          Hi Martijn, Thanks a lot for the answer. then if I have an old sync def or method, how can I change them into an asynchronous one?
          – user7936694
          Jul 26 '17 at 13:22












          You must either rewrite the function in async manner (there are many ready libs e.g. for pg, redis, http... mainly some io stuff) or use executors (mainly for use with cpu-intensive stuff) like ThreadPoolExecutro and ProcessPoolExecutor (docs.python.org/3/library/…). More examples can be found at pymotw.com/3/asyncio/executors.html
          – kwarunek
          Jul 26 '17 at 14:39






          You must either rewrite the function in async manner (there are many ready libs e.g. for pg, redis, http... mainly some io stuff) or use executors (mainly for use with cpu-intensive stuff) like ThreadPoolExecutro and ProcessPoolExecutor (docs.python.org/3/library/…). More examples can be found at pymotw.com/3/asyncio/executors.html
          – kwarunek
          Jul 26 '17 at 14:39














          @user7936694: exactly what kwarunek said; put code you can't make async in an executor, which then pretends your code is async by executing it under a different thread or in a separate process.
          – Martijn Pieters
          Jul 26 '17 at 14:48




          @user7936694: exactly what kwarunek said; put code you can't make async in an executor, which then pretends your code is async by executing it under a different thread or in a separate process.
          – Martijn Pieters
          Jul 26 '17 at 14:48












          Hi Martijn, thank a lot, seems it will take a while to change my old code into an asynchronous one :)
          – user7936694
          Jul 26 '17 at 19:01




          Hi Martijn, thank a lot, seems it will take a while to change my old code into an asynchronous one :)
          – user7936694
          Jul 26 '17 at 19:01


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f45319890%2fhow-does-the-asyncio-module-work-why-is-my-updated-sample-running-synchronously%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          What visual should I use to simply compare current year value vs last year in Power BI desktop

          How to ignore python UserWarning in pytest?

          Alexandru Averescu