Disable joblib.memory caching globally during unittest











up vote
0
down vote

favorite












I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.



Module1:



memory = Memory(location='/cache/')
@memory.cache
def heavy_function(...)
.....


Module2:



memory = Memory(location='/cache/')
@memory.cache
def heavy_function2(...)
.....


Module3:



import Module1
import Module2

def heavy_function3(...)
Module1.heavy_function1(...)
Module1.heavy_function1(...)
.....


Now I have a unit testing script and I want to disable the usage of the cache globally during the unit testing to make sure everything is correctly computed.
Is this possible without manually disabling it for each module via
Module1.memory.cachedir=None or without deleting the cachedir?



My current solution just patches each memory call manually



unittest1:



from joblib import Memory
import Module1
Module1.memory = Memory(location=None)
...
unittest.run()


unittest3:



from joblib import Memory
import Module1 # need to import module 1 just to disable its memory
import Module2 # need to import module 2 just to disable its memory
import Modul3
Module1.memory = Memory(location=None)
Module2.memory = Memory(location=None)
...
unittest.run()


The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.










share|improve this question
























  • why not monkey-patch those Memory instances?
    – georgexsh
    Nov 22 at 11:20










  • This is my current solution, but I thought maybe there is another way?
    – skjerns
    Nov 22 at 13:23















up vote
0
down vote

favorite












I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.



Module1:



memory = Memory(location='/cache/')
@memory.cache
def heavy_function(...)
.....


Module2:



memory = Memory(location='/cache/')
@memory.cache
def heavy_function2(...)
.....


Module3:



import Module1
import Module2

def heavy_function3(...)
Module1.heavy_function1(...)
Module1.heavy_function1(...)
.....


Now I have a unit testing script and I want to disable the usage of the cache globally during the unit testing to make sure everything is correctly computed.
Is this possible without manually disabling it for each module via
Module1.memory.cachedir=None or without deleting the cachedir?



My current solution just patches each memory call manually



unittest1:



from joblib import Memory
import Module1
Module1.memory = Memory(location=None)
...
unittest.run()


unittest3:



from joblib import Memory
import Module1 # need to import module 1 just to disable its memory
import Module2 # need to import module 2 just to disable its memory
import Modul3
Module1.memory = Memory(location=None)
Module2.memory = Memory(location=None)
...
unittest.run()


The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.










share|improve this question
























  • why not monkey-patch those Memory instances?
    – georgexsh
    Nov 22 at 11:20










  • This is my current solution, but I thought maybe there is another way?
    – skjerns
    Nov 22 at 13:23













up vote
0
down vote

favorite









up vote
0
down vote

favorite











I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.



Module1:



memory = Memory(location='/cache/')
@memory.cache
def heavy_function(...)
.....


Module2:



memory = Memory(location='/cache/')
@memory.cache
def heavy_function2(...)
.....


Module3:



import Module1
import Module2

def heavy_function3(...)
Module1.heavy_function1(...)
Module1.heavy_function1(...)
.....


Now I have a unit testing script and I want to disable the usage of the cache globally during the unit testing to make sure everything is correctly computed.
Is this possible without manually disabling it for each module via
Module1.memory.cachedir=None or without deleting the cachedir?



My current solution just patches each memory call manually



unittest1:



from joblib import Memory
import Module1
Module1.memory = Memory(location=None)
...
unittest.run()


unittest3:



from joblib import Memory
import Module1 # need to import module 1 just to disable its memory
import Module2 # need to import module 2 just to disable its memory
import Modul3
Module1.memory = Memory(location=None)
Module2.memory = Memory(location=None)
...
unittest.run()


The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.










share|improve this question















I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.



Module1:



memory = Memory(location='/cache/')
@memory.cache
def heavy_function(...)
.....


Module2:



memory = Memory(location='/cache/')
@memory.cache
def heavy_function2(...)
.....


Module3:



import Module1
import Module2

def heavy_function3(...)
Module1.heavy_function1(...)
Module1.heavy_function1(...)
.....


Now I have a unit testing script and I want to disable the usage of the cache globally during the unit testing to make sure everything is correctly computed.
Is this possible without manually disabling it for each module via
Module1.memory.cachedir=None or without deleting the cachedir?



My current solution just patches each memory call manually



unittest1:



from joblib import Memory
import Module1
Module1.memory = Memory(location=None)
...
unittest.run()


unittest3:



from joblib import Memory
import Module1 # need to import module 1 just to disable its memory
import Module2 # need to import module 2 just to disable its memory
import Modul3
Module1.memory = Memory(location=None)
Module2.memory = Memory(location=None)
...
unittest.run()


The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.







python unit-testing caching joblib






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 22 at 13:30

























asked Nov 15 at 11:36









skjerns

16611




16611












  • why not monkey-patch those Memory instances?
    – georgexsh
    Nov 22 at 11:20










  • This is my current solution, but I thought maybe there is another way?
    – skjerns
    Nov 22 at 13:23


















  • why not monkey-patch those Memory instances?
    – georgexsh
    Nov 22 at 11:20










  • This is my current solution, but I thought maybe there is another way?
    – skjerns
    Nov 22 at 13:23
















why not monkey-patch those Memory instances?
– georgexsh
Nov 22 at 11:20




why not monkey-patch those Memory instances?
– georgexsh
Nov 22 at 11:20












This is my current solution, but I thought maybe there is another way?
– skjerns
Nov 22 at 13:23




This is my current solution, but I thought maybe there is another way?
– skjerns
Nov 22 at 13:23












1 Answer
1






active

oldest

votes

















up vote
0
down vote













One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:



Module1



import os
memflag = os.environ.get('UNITTESTING', False)
memory = Memory(location= None if memflag else '/cache/')
@memory.cache
def heavy_function(...)
.....


unittest1



os.environ["UNITTESTING"] = '1'
import Module1
.....
unittest.run()
del os.environ["UNITTESTING"]





share|improve this answer























    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318600%2fdisable-joblib-memory-caching-globally-during-unittest%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes








    up vote
    0
    down vote













    One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:



    Module1



    import os
    memflag = os.environ.get('UNITTESTING', False)
    memory = Memory(location= None if memflag else '/cache/')
    @memory.cache
    def heavy_function(...)
    .....


    unittest1



    os.environ["UNITTESTING"] = '1'
    import Module1
    .....
    unittest.run()
    del os.environ["UNITTESTING"]





    share|improve this answer



























      up vote
      0
      down vote













      One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:



      Module1



      import os
      memflag = os.environ.get('UNITTESTING', False)
      memory = Memory(location= None if memflag else '/cache/')
      @memory.cache
      def heavy_function(...)
      .....


      unittest1



      os.environ["UNITTESTING"] = '1'
      import Module1
      .....
      unittest.run()
      del os.environ["UNITTESTING"]





      share|improve this answer

























        up vote
        0
        down vote










        up vote
        0
        down vote









        One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:



        Module1



        import os
        memflag = os.environ.get('UNITTESTING', False)
        memory = Memory(location= None if memflag else '/cache/')
        @memory.cache
        def heavy_function(...)
        .....


        unittest1



        os.environ["UNITTESTING"] = '1'
        import Module1
        .....
        unittest.run()
        del os.environ["UNITTESTING"]





        share|improve this answer














        One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:



        Module1



        import os
        memflag = os.environ.get('UNITTESTING', False)
        memory = Memory(location= None if memflag else '/cache/')
        @memory.cache
        def heavy_function(...)
        .....


        unittest1



        os.environ["UNITTESTING"] = '1'
        import Module1
        .....
        unittest.run()
        del os.environ["UNITTESTING"]






        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Nov 22 at 13:51

























        answered Nov 22 at 13:33









        skjerns

        16611




        16611






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318600%2fdisable-joblib-memory-caching-globally-during-unittest%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Catalogne

            Violoncelliste

            Héron pourpré