Getting authenticated programmatically in a form based authenticated webpage

Tweet
Most of the websites have form based authentication. You must enter your credentials in a form to see your data. Did a hacker in you ever wished to log in this website through a script, so that you can download all your data or make changes or post some data to your account? Well, here is a simple way to achieve it using python.

 

import urllib, urllib2

# Create a opendirector instance to manage your cookie handlers
urlopener=urllib2.build_opener(urllib2.HTTPCookieProcessor())

# Let us add chrome user agent to header to fool the website. Some website may deny access to non browser user-agent.
urlopener.addheaders = [('User-agent','Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.12 (KHTML, like Gecko) Chrome/9.0.576.0 Safari/534.12')]

# Let install the instance of opendirector
urllib2.install_opener(urlopener)

# Let us prepare the credentials. It may be important to find all possible hidden fields and fill it appropriately
credentials = {'email/userid':'yourUserId', 'password':'yourPassword', '_hiddenfield1' :'hiddenValue1', '_hiddenField2': 'hiddenValue2'}

# Let us post the urlencoded credentials to the credential check page (may or may not be same as login page)

# Note we are using installed urlopener to access this. This will automatically manage cookies
f=urlopener.open('http://secureWeb.site/login',urllib.urlencode(credentials))
s=f.read()

###### Process 's' - http response body as you want. ####
f.close()

# Access protected page with same urlopener
f=urlopener.open('http://secureWeb.site/ProtectedPage')
s=f.read()

##### Process 's' - http response body as you want. Check if you could really access protected page here #####
f.close()

I hope it helps and you enjoy accessing your favourite website through your script.


Tweet

Posting on Twitpic from commanline

Tweet

All of us like posting picture on twitter using twitpic. But most of us hate to go their website and do several clicks to reach upload page.  If you think similarly then here is command line method using curl to upload your pic. Linux, Unix and Mac users already have curl installed. Windows users must install curl utility to make this command work (or better switch to Ubuntu). 


Command -  
curl -F "username=TwitterUserId" --form-string "password=TwitterPassword" --form-string "message='MessageToTePosted'" -F "media=@yourPathToImage"  http://twitpic.com/api/uploadAndPost


where 
TwitterUserId is your Twitter User Id
TwitterPassword  is your Twitter Password
MessageToBePosted is message you want to post on twitter
yourPathToImage is path to the image

You can easily write a script to avoid typing this long command again and again. Also, if you want to upload multiple pictures at one time, writing a simple script might help.


Please note that twitpic uses http and not https for their api and thus your password might be at some risk. I wish they use https soon.



Tweet

Upload Workout to Dailymile

Tweet

Dailymile.com is one of the premier website to share your training with your friends. They provide apis to build application. I wrote a small code to upload my runkeeper data to dailymile.com. And this is how I did.

1. Register one application with dailymile.com as mentioned at http://www.dailymile.com/api/consumers/new.
2. Get access token as documented at http://www.dailymile.com/api/documentation/oauth.
3. Write a small code like following to extract data from a csv file and post it to dailymile.com using REST APIs as documented at http://www.dailymile.com/api/documentation

#!/usr/bin/python
from datetime import datetime, date, time
import httplib, urllib, string, datetime, sys


csvFile = open("test.csv", 'r')
for line in csvFile.readlines():
stringTokens = line.split(',')
timeTokens = stringTokens[4].split(':')
timetaken=(int(timeTokens[0])*60+int(timeTokens[1]))*60+int(timeTokens[2])
actDat = datetime.datetime.strptime(stringTokens[1], "%m-%d-%Y %H:%M:%S").strftime("%Y-%m-%d %H:%M:%S")
para = '{"oauth_token":"<Your Auth Code Goes here>", "message": "'+stringTokens[7].strip() + '", "workout": {"activity_type":"'+ string.lower(stringTokens[2]).strip() + '", "distance": {"value": ' + stringTokens[3] + ', "units":"kilometers"}, "duration": '+ str(timetaken) + ', "title":"' + stringTokens[1].strip() + '", "completed_at":"' + actDat +'" }}'
print para
headers = {"Accept": "application/json", "Content-type": "application/json"}


conn=httplib.HTTPSConnection("api.dailymile.com")
conn.request("POST","/entries.json", para, headers)
r1=conn.getresponse()
print r1.status, r1.reason
print r1.read()
conn.close()

If you find any problem / bug / issue, please let me know. I will be glad to help.  Please feel free to mail me at tachniki@gmail.com

Tweet

Exporting Runkeeper Data

Tweet

Runkeeper.com is one of the most well known website for tracking one's running, cycling etc. They have a wonderful website with charts and lots of interesting features. But as I mentioned here, I needed to export data from runkeeper and that is when I wrote a python script to download runkeeper data which I can import to dailymile.com.


If you need to export your data from runkeeper.com, you may use it too. This code comes with absolutely zero warranty. You may use it for your personal use. You can download  this script (runkeeper.py) from here.


This script can be executed by using following command
            ./runkeeper.py runkeeperUserId filename.csv
where runkeeperuserId is your Runkeeper User Id and filename is any name of any file where you want to store your activities.


Update December 10th 2010: 
----------------------------------------------------------------------------------
Now you can even download your private activities (along with your public activities) with a new script. You can download this script (runkeeper_wPrivAct.py) from here. 


This script can be executed by using following command
            ./runkeeper_wPrivAct.py emailId filename.csv
where emailId is your email id associated to runkeeper and filename is name of any file where you want to store your activities.


You will need to enter your runkeeper password when asked for. Please note that entering email id and password is as safe as entering on runkeeper website. This script doesn't store or send your password anywhere except using it to log you onto runkeeper website to download your data.. 
--------------------------------------------------------------------------------


These script will produce a comma separated text file in following format:

If you find any problem / bug / issue, please let me know. I will be glad to help.  Please feel free to mail me at tachniki@gmail.com

Tweet