Why would using Selenium webdriver to execute JS fine locally on my Mac, but not in a Docker container?











up vote
1
down vote

favorite












Docker environment: Docker Container, ubuntu:18.10, geckodriver-v0.23.0-linux64, selenium-3.14.1



Local environment: Mac OSX, geckodriver-v0.23.0-linux64, selenium-3.14.1



Why would executing JS work fine locally on my Mac, but not in a Docker container?



Python function that is triggering error on nfl.com:



def scroll_down(driver, value):
""" Scroll down some """
driver.execute_script("window.scrollBy(0,"+str(value)+")")


Traceback from nfl.com



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 150, in main
detect_full_html_loaded(driver)
File "/app/foo/foo.py", line 55, in detect_full_html_loaded
scroll_down(driver, 1000)
File "/app/foo/foo.py", line 45, in scroll_down
driver.execute_script("window.scrollBy(0,"+str(value)+")")
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 636, in execute_script
'args': converted_args})['value']
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette


Python function that is triggering error on nytimes.com and sony.com:



def capture_screenshot(assets_dir, url, sleep, driver):
""" Take simple screenshot of above-the-fold """
driver.get(url)
time.sleep(sleep)
driver.set_window_size(1400, 700)
Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
print('Created: "' + assets_dir + 'screenshot.png' + '"')


Traceback on nytimes.com:



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 152, in main
capture_element_pic(input_file, assets_dir, data['finalUrl'], sel, sleep, driver)
File "/app/foo/foo.py", line 89, in capture_element_pic
driver.set_window_size(1400, driver.execute_script("return document.body.parentNode.scrollHeight"))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1090, in set_window_size
self.set_window_rect(width=int(width), height=int(height))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1182, in set_window_rect
"height": height})['value']
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: [Exception... "Failure" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://marionette/content/browser.js :: get rect :: line 254" data: no]


Traceback from sony.com



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 149, in main
capture_screenshot(assets_dir, data['finalUrl'], sleep, driver)
File "/app/foo/foo.py", line 82, in capture_screenshot
Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1064, in get_screenshot_as_png
return base64.b64decode(self.get_screenshot_as_base64().encode('ascii'))
AttributeError: 'NoneType' object has no attribute 'encode'


Again, this works fine locally.










share|improve this question
























  • Considering exception logs it doesn't seem like website is rejecting your stuff because it is Selenium
    – Andersson
    11 hours ago










  • Forgot to mention that I am running this in a Docker container. And its seems to work when I run it natively on my Mac. So maybe its actually a question for the Docker people.
    – heart.cooks.mind
    11 hours ago






  • 1




    This might be configuration/performance issue...
    – Andersson
    11 hours ago










  • Is it possible for a site to detect that you're using selenium? They can detect which browser you're using which is the geckodriver but I can't see how they would know if you're controlling the driver via selenium or not.
    – Marcel Wilson
    9 hours ago










  • I think @Anderson is correct I think this ended up being a Docker issue related to configuration/performance, since I could make it work locally but not in the container. So I am going to pursue it in that direction and close this issue.
    – heart.cooks.mind
    9 hours ago















up vote
1
down vote

favorite












Docker environment: Docker Container, ubuntu:18.10, geckodriver-v0.23.0-linux64, selenium-3.14.1



Local environment: Mac OSX, geckodriver-v0.23.0-linux64, selenium-3.14.1



Why would executing JS work fine locally on my Mac, but not in a Docker container?



Python function that is triggering error on nfl.com:



def scroll_down(driver, value):
""" Scroll down some """
driver.execute_script("window.scrollBy(0,"+str(value)+")")


Traceback from nfl.com



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 150, in main
detect_full_html_loaded(driver)
File "/app/foo/foo.py", line 55, in detect_full_html_loaded
scroll_down(driver, 1000)
File "/app/foo/foo.py", line 45, in scroll_down
driver.execute_script("window.scrollBy(0,"+str(value)+")")
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 636, in execute_script
'args': converted_args})['value']
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette


Python function that is triggering error on nytimes.com and sony.com:



def capture_screenshot(assets_dir, url, sleep, driver):
""" Take simple screenshot of above-the-fold """
driver.get(url)
time.sleep(sleep)
driver.set_window_size(1400, 700)
Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
print('Created: "' + assets_dir + 'screenshot.png' + '"')


Traceback on nytimes.com:



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 152, in main
capture_element_pic(input_file, assets_dir, data['finalUrl'], sel, sleep, driver)
File "/app/foo/foo.py", line 89, in capture_element_pic
driver.set_window_size(1400, driver.execute_script("return document.body.parentNode.scrollHeight"))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1090, in set_window_size
self.set_window_rect(width=int(width), height=int(height))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1182, in set_window_rect
"height": height})['value']
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: [Exception... "Failure" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://marionette/content/browser.js :: get rect :: line 254" data: no]


Traceback from sony.com



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 149, in main
capture_screenshot(assets_dir, data['finalUrl'], sleep, driver)
File "/app/foo/foo.py", line 82, in capture_screenshot
Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1064, in get_screenshot_as_png
return base64.b64decode(self.get_screenshot_as_base64().encode('ascii'))
AttributeError: 'NoneType' object has no attribute 'encode'


Again, this works fine locally.










share|improve this question
























  • Considering exception logs it doesn't seem like website is rejecting your stuff because it is Selenium
    – Andersson
    11 hours ago










  • Forgot to mention that I am running this in a Docker container. And its seems to work when I run it natively on my Mac. So maybe its actually a question for the Docker people.
    – heart.cooks.mind
    11 hours ago






  • 1




    This might be configuration/performance issue...
    – Andersson
    11 hours ago










  • Is it possible for a site to detect that you're using selenium? They can detect which browser you're using which is the geckodriver but I can't see how they would know if you're controlling the driver via selenium or not.
    – Marcel Wilson
    9 hours ago










  • I think @Anderson is correct I think this ended up being a Docker issue related to configuration/performance, since I could make it work locally but not in the container. So I am going to pursue it in that direction and close this issue.
    – heart.cooks.mind
    9 hours ago













up vote
1
down vote

favorite









up vote
1
down vote

favorite











Docker environment: Docker Container, ubuntu:18.10, geckodriver-v0.23.0-linux64, selenium-3.14.1



Local environment: Mac OSX, geckodriver-v0.23.0-linux64, selenium-3.14.1



Why would executing JS work fine locally on my Mac, but not in a Docker container?



Python function that is triggering error on nfl.com:



def scroll_down(driver, value):
""" Scroll down some """
driver.execute_script("window.scrollBy(0,"+str(value)+")")


Traceback from nfl.com



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 150, in main
detect_full_html_loaded(driver)
File "/app/foo/foo.py", line 55, in detect_full_html_loaded
scroll_down(driver, 1000)
File "/app/foo/foo.py", line 45, in scroll_down
driver.execute_script("window.scrollBy(0,"+str(value)+")")
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 636, in execute_script
'args': converted_args})['value']
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette


Python function that is triggering error on nytimes.com and sony.com:



def capture_screenshot(assets_dir, url, sleep, driver):
""" Take simple screenshot of above-the-fold """
driver.get(url)
time.sleep(sleep)
driver.set_window_size(1400, 700)
Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
print('Created: "' + assets_dir + 'screenshot.png' + '"')


Traceback on nytimes.com:



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 152, in main
capture_element_pic(input_file, assets_dir, data['finalUrl'], sel, sleep, driver)
File "/app/foo/foo.py", line 89, in capture_element_pic
driver.set_window_size(1400, driver.execute_script("return document.body.parentNode.scrollHeight"))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1090, in set_window_size
self.set_window_rect(width=int(width), height=int(height))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1182, in set_window_rect
"height": height})['value']
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: [Exception... "Failure" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://marionette/content/browser.js :: get rect :: line 254" data: no]


Traceback from sony.com



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 149, in main
capture_screenshot(assets_dir, data['finalUrl'], sleep, driver)
File "/app/foo/foo.py", line 82, in capture_screenshot
Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1064, in get_screenshot_as_png
return base64.b64decode(self.get_screenshot_as_base64().encode('ascii'))
AttributeError: 'NoneType' object has no attribute 'encode'


Again, this works fine locally.










share|improve this question















Docker environment: Docker Container, ubuntu:18.10, geckodriver-v0.23.0-linux64, selenium-3.14.1



Local environment: Mac OSX, geckodriver-v0.23.0-linux64, selenium-3.14.1



Why would executing JS work fine locally on my Mac, but not in a Docker container?



Python function that is triggering error on nfl.com:



def scroll_down(driver, value):
""" Scroll down some """
driver.execute_script("window.scrollBy(0,"+str(value)+")")


Traceback from nfl.com



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 150, in main
detect_full_html_loaded(driver)
File "/app/foo/foo.py", line 55, in detect_full_html_loaded
scroll_down(driver, 1000)
File "/app/foo/foo.py", line 45, in scroll_down
driver.execute_script("window.scrollBy(0,"+str(value)+")")
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 636, in execute_script
'args': converted_args})['value']
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette


Python function that is triggering error on nytimes.com and sony.com:



def capture_screenshot(assets_dir, url, sleep, driver):
""" Take simple screenshot of above-the-fold """
driver.get(url)
time.sleep(sleep)
driver.set_window_size(1400, 700)
Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
print('Created: "' + assets_dir + 'screenshot.png' + '"')


Traceback on nytimes.com:



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 152, in main
capture_element_pic(input_file, assets_dir, data['finalUrl'], sel, sleep, driver)
File "/app/foo/foo.py", line 89, in capture_element_pic
driver.set_window_size(1400, driver.execute_script("return document.body.parentNode.scrollHeight"))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1090, in set_window_size
self.set_window_rect(width=int(width), height=int(height))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1182, in set_window_rect
"height": height})['value']
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: [Exception... "Failure" nsresult: "0x80004005 (NS_ERROR_FAILURE)" location: "JS frame :: chrome://marionette/content/browser.js :: get rect :: line 254" data: no]


Traceback from sony.com



Traceback (most recent call last):
File "/app/foo/foo.py", line 159, in <module>
main()
File "/app/foo/foo.py", line 149, in main
capture_screenshot(assets_dir, data['finalUrl'], sleep, driver)
File "/app/foo/foo.py", line 82, in capture_screenshot
Image.open(BytesIO(driver.get_screenshot_as_png())).save(os.path.join(assets_dir,'screenshot.png'))
File "/usr/local/lib/python3.6/dist-packages/selenium/webdriver/remote/webdriver.py", line 1064, in get_screenshot_as_png
return base64.b64decode(self.get_screenshot_as_base64().encode('ascii'))
AttributeError: 'NoneType' object has no attribute 'encode'


Again, this works fine locally.







python selenium docker selenium-webdriver






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 9 hours ago

























asked 11 hours ago









heart.cooks.mind

442214




442214












  • Considering exception logs it doesn't seem like website is rejecting your stuff because it is Selenium
    – Andersson
    11 hours ago










  • Forgot to mention that I am running this in a Docker container. And its seems to work when I run it natively on my Mac. So maybe its actually a question for the Docker people.
    – heart.cooks.mind
    11 hours ago






  • 1




    This might be configuration/performance issue...
    – Andersson
    11 hours ago










  • Is it possible for a site to detect that you're using selenium? They can detect which browser you're using which is the geckodriver but I can't see how they would know if you're controlling the driver via selenium or not.
    – Marcel Wilson
    9 hours ago










  • I think @Anderson is correct I think this ended up being a Docker issue related to configuration/performance, since I could make it work locally but not in the container. So I am going to pursue it in that direction and close this issue.
    – heart.cooks.mind
    9 hours ago


















  • Considering exception logs it doesn't seem like website is rejecting your stuff because it is Selenium
    – Andersson
    11 hours ago










  • Forgot to mention that I am running this in a Docker container. And its seems to work when I run it natively on my Mac. So maybe its actually a question for the Docker people.
    – heart.cooks.mind
    11 hours ago






  • 1




    This might be configuration/performance issue...
    – Andersson
    11 hours ago










  • Is it possible for a site to detect that you're using selenium? They can detect which browser you're using which is the geckodriver but I can't see how they would know if you're controlling the driver via selenium or not.
    – Marcel Wilson
    9 hours ago










  • I think @Anderson is correct I think this ended up being a Docker issue related to configuration/performance, since I could make it work locally but not in the container. So I am going to pursue it in that direction and close this issue.
    – heart.cooks.mind
    9 hours ago
















Considering exception logs it doesn't seem like website is rejecting your stuff because it is Selenium
– Andersson
11 hours ago




Considering exception logs it doesn't seem like website is rejecting your stuff because it is Selenium
– Andersson
11 hours ago












Forgot to mention that I am running this in a Docker container. And its seems to work when I run it natively on my Mac. So maybe its actually a question for the Docker people.
– heart.cooks.mind
11 hours ago




Forgot to mention that I am running this in a Docker container. And its seems to work when I run it natively on my Mac. So maybe its actually a question for the Docker people.
– heart.cooks.mind
11 hours ago




1




1




This might be configuration/performance issue...
– Andersson
11 hours ago




This might be configuration/performance issue...
– Andersson
11 hours ago












Is it possible for a site to detect that you're using selenium? They can detect which browser you're using which is the geckodriver but I can't see how they would know if you're controlling the driver via selenium or not.
– Marcel Wilson
9 hours ago




Is it possible for a site to detect that you're using selenium? They can detect which browser you're using which is the geckodriver but I can't see how they would know if you're controlling the driver via selenium or not.
– Marcel Wilson
9 hours ago












I think @Anderson is correct I think this ended up being a Docker issue related to configuration/performance, since I could make it work locally but not in the container. So I am going to pursue it in that direction and close this issue.
– heart.cooks.mind
9 hours ago




I think @Anderson is correct I think this ended up being a Docker issue related to configuration/performance, since I could make it work locally but not in the container. So I am going to pursue it in that direction and close this issue.
– heart.cooks.mind
9 hours ago

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53416143%2fwhy-would-using-selenium-webdriver-to-execute-js-fine-locally-on-my-mac-but-not%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53416143%2fwhy-would-using-selenium-webdriver-to-execute-js-fine-locally-on-my-mac-but-not%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Trompette piccolo

Slow SSRS Report in dynamic grouping and multiple parameters

Simon Yates (cyclisme)