Disable joblib.memory caching globally during unittest
up vote
0
down vote
favorite
I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.
Module1:
memory = Memory(location='/cache/')
@memory.cache
def heavy_function(...)
.....
Module2:
memory = Memory(location='/cache/')
@memory.cache
def heavy_function2(...)
.....
Module3:
import Module1
import Module2
def heavy_function3(...)
Module1.heavy_function1(...)
Module1.heavy_function1(...)
.....
Now I have a unit testing script and I want to disable the usage of the cache globally during the unit testing to make sure everything is correctly computed.
Is this possible without manually disabling it for each module via
Module1.memory.cachedir=None or without deleting the cachedir?
My current solution just patches each memory call manually
unittest1:
from joblib import Memory
import Module1
Module1.memory = Memory(location=None)
...
unittest.run()
unittest3:
from joblib import Memory
import Module1 # need to import module 1 just to disable its memory
import Module2 # need to import module 2 just to disable its memory
import Modul3
Module1.memory = Memory(location=None)
Module2.memory = Memory(location=None)
...
unittest.run()
The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.
python unit-testing caching joblib
add a comment |
up vote
0
down vote
favorite
I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.
Module1:
memory = Memory(location='/cache/')
@memory.cache
def heavy_function(...)
.....
Module2:
memory = Memory(location='/cache/')
@memory.cache
def heavy_function2(...)
.....
Module3:
import Module1
import Module2
def heavy_function3(...)
Module1.heavy_function1(...)
Module1.heavy_function1(...)
.....
Now I have a unit testing script and I want to disable the usage of the cache globally during the unit testing to make sure everything is correctly computed.
Is this possible without manually disabling it for each module via
Module1.memory.cachedir=None or without deleting the cachedir?
My current solution just patches each memory call manually
unittest1:
from joblib import Memory
import Module1
Module1.memory = Memory(location=None)
...
unittest.run()
unittest3:
from joblib import Memory
import Module1 # need to import module 1 just to disable its memory
import Module2 # need to import module 2 just to disable its memory
import Modul3
Module1.memory = Memory(location=None)
Module2.memory = Memory(location=None)
...
unittest.run()
The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.
python unit-testing caching joblib
why not monkey-patch thoseMemoryinstances?
– georgexsh
Nov 22 at 11:20
This is my current solution, but I thought maybe there is another way?
– skjerns
Nov 22 at 13:23
add a comment |
up vote
0
down vote
favorite
up vote
0
down vote
favorite
I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.
Module1:
memory = Memory(location='/cache/')
@memory.cache
def heavy_function(...)
.....
Module2:
memory = Memory(location='/cache/')
@memory.cache
def heavy_function2(...)
.....
Module3:
import Module1
import Module2
def heavy_function3(...)
Module1.heavy_function1(...)
Module1.heavy_function1(...)
.....
Now I have a unit testing script and I want to disable the usage of the cache globally during the unit testing to make sure everything is correctly computed.
Is this possible without manually disabling it for each module via
Module1.memory.cachedir=None or without deleting the cachedir?
My current solution just patches each memory call manually
unittest1:
from joblib import Memory
import Module1
Module1.memory = Memory(location=None)
...
unittest.run()
unittest3:
from joblib import Memory
import Module1 # need to import module 1 just to disable its memory
import Module2 # need to import module 2 just to disable its memory
import Modul3
Module1.memory = Memory(location=None)
Module2.memory = Memory(location=None)
...
unittest.run()
The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.
python unit-testing caching joblib
I use the joblib.Memory module to cache some functions within several modules. The cache is initialized within modules and classes separately.
Module1:
memory = Memory(location='/cache/')
@memory.cache
def heavy_function(...)
.....
Module2:
memory = Memory(location='/cache/')
@memory.cache
def heavy_function2(...)
.....
Module3:
import Module1
import Module2
def heavy_function3(...)
Module1.heavy_function1(...)
Module1.heavy_function1(...)
.....
Now I have a unit testing script and I want to disable the usage of the cache globally during the unit testing to make sure everything is correctly computed.
Is this possible without manually disabling it for each module via
Module1.memory.cachedir=None or without deleting the cachedir?
My current solution just patches each memory call manually
unittest1:
from joblib import Memory
import Module1
Module1.memory = Memory(location=None)
...
unittest.run()
unittest3:
from joblib import Memory
import Module1 # need to import module 1 just to disable its memory
import Module2 # need to import module 2 just to disable its memory
import Modul3
Module1.memory = Memory(location=None)
Module2.memory = Memory(location=None)
...
unittest.run()
The more modules I create, the more manual patching up of the Memory I need. I thought there might be a better solution. One work-around is proposed by me below.
python unit-testing caching joblib
python unit-testing caching joblib
edited Nov 22 at 13:30
asked Nov 15 at 11:36
skjerns
16611
16611
why not monkey-patch thoseMemoryinstances?
– georgexsh
Nov 22 at 11:20
This is my current solution, but I thought maybe there is another way?
– skjerns
Nov 22 at 13:23
add a comment |
why not monkey-patch thoseMemoryinstances?
– georgexsh
Nov 22 at 11:20
This is my current solution, but I thought maybe there is another way?
– skjerns
Nov 22 at 13:23
why not monkey-patch those
Memory instances?– georgexsh
Nov 22 at 11:20
why not monkey-patch those
Memory instances?– georgexsh
Nov 22 at 11:20
This is my current solution, but I thought maybe there is another way?
– skjerns
Nov 22 at 13:23
This is my current solution, but I thought maybe there is another way?
– skjerns
Nov 22 at 13:23
add a comment |
1 Answer
1
active
oldest
votes
up vote
0
down vote
One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:
Module1
import os
memflag = os.environ.get('UNITTESTING', False)
memory = Memory(location= None if memflag else '/cache/')
@memory.cache
def heavy_function(...)
.....
unittest1
os.environ["UNITTESTING"] = '1'
import Module1
.....
unittest.run()
del os.environ["UNITTESTING"]
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
0
down vote
One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:
Module1
import os
memflag = os.environ.get('UNITTESTING', False)
memory = Memory(location= None if memflag else '/cache/')
@memory.cache
def heavy_function(...)
.....
unittest1
os.environ["UNITTESTING"] = '1'
import Module1
.....
unittest.run()
del os.environ["UNITTESTING"]
add a comment |
up vote
0
down vote
One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:
Module1
import os
memflag = os.environ.get('UNITTESTING', False)
memory = Memory(location= None if memflag else '/cache/')
@memory.cache
def heavy_function(...)
.....
unittest1
os.environ["UNITTESTING"] = '1'
import Module1
.....
unittest.run()
del os.environ["UNITTESTING"]
add a comment |
up vote
0
down vote
up vote
0
down vote
One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:
Module1
import os
memflag = os.environ.get('UNITTESTING', False)
memory = Memory(location= None if memflag else '/cache/')
@memory.cache
def heavy_function(...)
.....
unittest1
os.environ["UNITTESTING"] = '1'
import Module1
.....
unittest.run()
del os.environ["UNITTESTING"]
One work-around is to set a flag or an environment variable when running tests. Then check for these flags before initializing the Memory:
Module1
import os
memflag = os.environ.get('UNITTESTING', False)
memory = Memory(location= None if memflag else '/cache/')
@memory.cache
def heavy_function(...)
.....
unittest1
os.environ["UNITTESTING"] = '1'
import Module1
.....
unittest.run()
del os.environ["UNITTESTING"]
edited Nov 22 at 13:51
answered Nov 22 at 13:33
skjerns
16611
16611
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318600%2fdisable-joblib-memory-caching-globally-during-unittest%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
why not monkey-patch those
Memoryinstances?– georgexsh
Nov 22 at 11:20
This is my current solution, but I thought maybe there is another way?
– skjerns
Nov 22 at 13:23