Tag Archives

14 Articles
Posted by Dave on

Browser Scripting

Browser Scripting

I discovered that my notes on capturing the ello feed were out of date and have corrected the page in that the stuff about ello is now useless. This page holds the general notes I made about web scripting. They may be equally out of date but I am not yet ready to delete them. My [old] notes are below/overleaf.

Posted by Dave on

Unicode in Python

Unicode in Python

It’s a very long time since I have had to deal with binary code translations, but when dealing with HTML and JSON with Python, I need to get to grips with it. The € symbol for instance requires the extended character set support and rss should be written in utf-8. This page collects the notes I made in learning about Python’s support for unicode/ascii translations.

Posted by Dave on

Web Site Testing with MaxQ

Web Site Testing with MaxQ

MAXQ is a web proxy with recording and replay features. It is designed to capture and replay tests. They are quite keen that one understands the terms of use of target sites since many prohibit automated access and these days it would seem defend themselves against it.  This article documents how to run MaxQ and make tests, but not how to successfully run them. (If that’s why you’ve come here, look elsewhere but if you have answers please comment here. )

Posted by Dave on

Converting dates to strings using Python

Converting dates to strings using Python

While working on converting ello.json to rss, I needed to convert the date formats, I needed to do it in Python and found the following code worked, using strptime and strfmtime. I had to augment the date string with the name of the day of the week attribute, so felt it had to via a date data type. (This is an important part of the requirement, since otherwise string manipulation would have been enough.) Finding exampes was hard, so I hope it helps,

Posted by Dave on

JSON to XML

JSON to XML

In order to export the ello.co feed, I need to convert the JSON to XML. This isn’t the first time I have needed to look at this sort of problem and so I have created a page to hold my notes. I have put these things together in a python script, and posted it to github, in a repo called ello2rss.

Posted by Dave on

WAMP

WAMP

I needed to install WAMP on my PC, to support some Amazon developments; I needed a screen interface to my database. Here are some notes and links. I talk about installation, and port conflicts. I might expand the install notes to include advice on mysql security.

Posted by Dave on

delicious linkrolls

Time to revise this page. It was originally created to document the runes on how to use the delicious provided javascript linkroll generator, it then covered the slow and painful death of the service and my attempts to postpone the inevitable.  In late 2019, I found my backup copy and posted it to the blog,

https://davelevy.info/is-this-my-delicious-linkroll/

There are some problems with this, most obviously the tags are hidden. I have written some code in python, to parse the html and create a python dictionary, and also some code to build a tag index. I have some code to see if the linked sites are still active but there’s over 3000 bookmarks at it takes some time, and doesn’t distinguish between those that have gone away and those that are being resold. (I need to go through them for continued relevance also.)

The links are held in a pickle database, which could easily be converted to XML or JSON. I have code to delete a record on the basis of a numeric surrogate key. I can build two indices, on has the key and the tags, held as a string, the other has a list of key-tag pairs.

  • Document which sites no linger work and/or are no longer relevant.
  • Define the output formats, html, rss or an active query and if the latter how to host it.
  • Document the delicious date format field
  • Write some restartable HTML status checking code.
  • Amend the index generators ro  write to a file, so there is some persistence

The slow death of delicious’ rss feeds

I originally saved the javascript linkroll code as text in a page, and latterly, from Oct 2013 used Artiss Embeded Code and I located the scripts in code objects. delicious withdrew the javascript. I implemented JP’s Get RSS Feed to fix the withdrawal of the delicious javascript;

I rendered the feed using wordpress. At some time delicious changed their URL from delicious.com back to del.icio.us. At the same time, they decided to insert an advert into the first item of the feed.

This has led to all my bookmark feeds in this wiki breaking, and to asking the question as to whether one wants to fix it as I no longer control the content. I have posted to the wordpress support site to implement a “start from” parameter. I wonder if CSS display:none; properties could be used to supress the first element in the feed.

This wiki has a number of pages which used the del.icio.us feed to expand the articles and to point at my bookmarks. I haven’t decided whether to fix them or not.

I have asked that the plugin is adjusted to exclude the first element and I propose to examine planet to see if I can put a filter into the consumption code lines. See https://davelevy.info/wiki/planet-planet/ for my planet documentation.

See https://davelevy.info/wiki/rendering-rss-on-my-blog-or-wiki/ for my jp-rss-feed documentation and https://wordpress.org/plugins/jps-get-rss-feed/ for Jimmy’s documentation.

ooOOOoo

Posted by Dave on

Twitter API V1.1 (Deprecated)

Twitter API V1.1 (Deprecated)

This post was written in 2013 and I discovered in May 2018 that most of the links failed. It’s been a while, and it surprises me that my little wiki has lasted longer than the resources I point to, but not here. I could have deleted this post, but chose not to; I have heavily edited it so it’s as much a diary as the technical note it once was.

Posted by Dave on

Planet Planet

I started using Planet in about 2008 2006, it would seem? I rapidly came too “Planet Venus” and this article was originally written during that time, and was copied from the snipsnap wiki, itself copied from my sun blog. This article focuses on installation and administration. It includes links to my scripts. It was heavily amended in June 2020 in thanks to the location of my tools and Sam’s code being on github.

Posted by Dave on

Python

Python

I returned to Python in Oct 2010.

I have two problems to solve,

  1. converting my bliki to wordpress, (I never used Python for this)
  2. scripting for KDB & Hadoop

This article contains notes started in 2011, I did not make this an omnibus entry and it is very sketchy, covering file I/O, date & time, dictionaries vs. lists and PYTHONPATH.

Posted by Dave on

Python and datetime

Links

Here are some links

Examples

An example of using datetime is

import time
import datetime
now=datetime.datetime.now() 

#now.tzinfo = 'GMT'; # fails, now.tzinfo is immutable

print now.ctime() print now.isoformat() print now.tzinfo

but otherwise produces the following output

Fri Jan  2 15:15:07 2009
2009-01-02T15:15:07.896000
None

I performed these tests on a windows XP box. tzinfo is empty is this a windows/configuration thing? I can test this with UNIX. I tried setting the TZ variable before running the program and it makes no difference. I found one reference that says Python datetime doesn’t do TimeZones.

Obviously, using the from statement would change the call syntax

from datetime import datetime
now=datetime.now()

should work