I discovered that my notes on capturing the ello feed were out of date and have corrected the page in that the stuff about ello is now useless. This page holds the general notes I made about web scripting. They may be equally out of date but I am not yet ready to delete them. My [old] notes are below/overleaf.
I am considering a scripting solution, first Python,
- http://stackoverflow.com/questions/3876070/how-to-script-firefox-or-any-mozilla-based-browser
- https://docs.python.org/3/howto/urllib2.html
Some detailed examples using python tools
- http://stackoverflow.com/questions/2910221/how-can-i-login-to-a-website-with-python
- http://stackoverflow.com/questions/8316818/login-to-website-using-python
- http://stackoverflow.com/questions/13925983/login-to-website-using-urllib2-python-2-7
- http://stackoverflow.com/questions/2910221/how-can-i-login-to-a-website-with-python mentions package requests
For more checkout, this google search, “web login using python”
however the first of these links, contains proposals pointing at alternative scripting, using Selenium, mechanize & twill.
- http://docs.seleniumhq.org/projects/ide/
- http://wwwsearch.sourceforge.net/mechanize/
- http://twill.idyll.org/
I have tried twill, the login is tricky, and not well documented, or I am too stupid. the form hosting page, http://elllo.co/enter has form entry field names containing [], and something isn’t working. (It refuses to recognise the form fields.) ello detects the twill go statement and places a captcha/unblock request screen in the way. Do I continue down this path or shall I look at the at the basic python libraries.
I was also pointed at MaxQ, an automated web site test harness, for more see https://davelevy.info/wiki/web-site-testing-with-maxq/ but I havn’t got this to work.
I found the reference to eautiful Soap a python based document parser, helpful, may be for identifying the document structure and thus the arguments to twill
- https://docs.python.org/2/howto/urllib2.html
- http://www.voidspace.org.uk/python/articles/authentication.shtml
In some of my reading I was advised to look at Tamper Data,