Browsing articles in "hacks"

uop2cal: Subscribe to your @PortsmouthUni timetable as an iCal feed (UPyoursUPlink?)

Anyone who goes to the University of Portsmouth will be familiar with UPlink, the University’s portal system, and with the timetable it contains. Most people (especially anyone under what used to be ECE) will be familiar with the frustration of lectures changing times/dates/moving room/no longer existing despite being marked as “confirmed”, and the univeristy-standard oh-so-helpful response of “You should be checking it everyday”, along with the other dozen-or-so university sites we’re supposed to be checking every day. (Can you notice my frustration yet? I was originally planning to call this UPyoursUPlink but have since decided on the less-offensive but less-descriptive uop2cal)

This script is a pretty simple one, it uses cURL to screen-scrape the portal and retrieve your timetable, an obnoxious set of regexes (<3) to parse the data into an array, and then the iCalcreator class to generate an ICS file of your timetable, complete with the details (name, lecturer, room, confirmed/unconfirmed (not that I can find a difference, mind), group, etc).

Update: Previously, I was supplying a hosted copy for anyone to use, but I have been advised by the University that you willingly giving me your username and password somehow contravenes the Computer Misuse Act 1990. Whether or not this is the case is irrelevant, as they have asked I stopped doing this. In any case, the source code is still available below (unless they later tell me that’s a problem somehow, too), so you can run your own copy.

Update update: Apparently the University still weren’t too happy with the provisions I had made the last time they contacted me, and have once again requested I remove this. As such, this source is no longer publicly available. Queries about why can be directed towards the University itself, queries about the source are welcome to me.

tinyurlfs, file2tinyurl & tinyurl2file

IRC Log (or, initial idea conception)

<Ignite> :)
<cmantito> ok, this is gonna sound crazy
<cmantito> really, really crazy
<cmantito> as you might have (or not) seen when it was in the /t, I wrote a set of scripts that could encode files
as URLs to be stored in a tinyurl, and then extracted again
<LeaChim> yes, that was quite cool
<cmantito> now, I'm writing a mountable filesystem, when you copy a file into it, it uploads it to tinyurl, grabs
the tinyurlid, and passes it to a PHP API running on my server and it stores the filename and the tinyurl ID
<LeaChim> why a php script?
<LeaChim> why can't it store this locally?
<cmantito> I'm getting there ;)
<cmantito> and when you read a file, it queries the API running on my server with the filename, gets the urlID, and
allows you to open the file in place
<cmantito> ideally, it's a write-once filesystem, you can put files into it, but not remove them, overwrite them, or
modify them.
<cmantito> the reason for using a remote API for it, is so that it's a unified filesystem
<cmantito> ie, I copy a file into it, and you can copy it right back out again
<cmantito> it's a giant filestore!
<LeaChim> heh
<cmantito> for *everyone*
<LeaChim> quite cool
<cmantito> and
<cmantito> if someone wants to bitch at me for "storing" files of naughty materials
<cmantito> I can use the bittorrent defense
<cmantito> "I only store metadata"
<Ignite> rofl
<LeaChim> heh
<LeaChim> well, you'll be implemented this as fuse right?
<cmantito> yes.
<cmantito> it's half-done
<Ignite> cmantito, so like
<Ignite> cmantito, we can all use it?
<cmantito> yes :D
<Ignite> So we can sort of share warez and shits?
<Ignite> :x
<cmantito> if the links are already in the database, it can read them and list them
<LeaChim> cmantito, for rmdir unlink rename chmod chown truncate you should just return permission denied
<cmantito> LeaChim: exactly! :D
<cmantito> fserr(13)
<cmantito> ENOACCES
<LeaChim> so all you've got left are mkdir symlink link utime, and file creation
<cmantito> well those 4 are unnecessary
<LeaChim> in fact, you might as well not have symlink or link
<LeaChim> no, keep mkdir
<cmantito> exactly
<cmantito> no, it's a flat filesystem
<Ignite> No
<LeaChim> boring
<Ignite> mkdir
<LeaChim> and horrible to list
<Ignite> WE WANT MKDIR
<LeaChim> add mkdir
* Ignite chants
<cmantito> ffs *fine*
<LeaChim> besides, mkdir is just php side
<Ignite> Rofl
<cmantito> now it's gonne take even longer.
<Ignite> xD
<cmantito> I *was* trying to keep it relatively simple
<cmantito> since if it takes off, tinyurl will kinda notice
<cmantito> Upload a DVD image "What's that, a 4.93921239x10^9 character URL!?"
<cmantito> actaully more
<cmantito> uuencode pads.
<cmantito> butokfine
<LeaChim> cmantito, you might want to add some code to allow you to split up files
<cmantito> I'll put in the ability to add one level of directories.
<cmantito> ONE XD
<LeaChim> lol
<LeaChim> why not more than one?
<LeaChim> it's just going to get horrible to list directories ;
<LeaChim> and maybe you should add something to expire things, so the directory listing doesn't end up being
gigantic
<cmantito> I'm trying to limit the amount of HTTP queries I have to make for any given requests.
<cmantito> s/(request)s/\1/

Original Scripts

The original file2tinyurl and tinyurl2file scripts were originally published at RackLoad (now defunct) and then at the Xelix wiki, followed by my own CodeWiki to continue development into a FUSE mountable filesystem. They now live here, source available below.

Development Roadmap

It actually works. By using file2tinyurl, and then putting the relevant values into the database manually. the files are accessible. It is not directory-structured yet, and file upload isn’t complete yet either. Please keep adding feature requests, and as soon as a usable version (even without ALL the features) is alpha, I’ll chuck it up here.

Original Proof of Concept Source

These scripts requires the following Perl modules:

  • LWP::UserAgent
  • Convert::UU
  • HTML::Entities
  • URI::Escape

These modules all have dependencies of their own. To easily install the modules and all their dependencies, execute the following command as root and it will prompt you to install the dependencies. Pressing return repeatedly will generally suffice.

cpan install LWP::UserAgent Convert::UU HTML::Entities URI::Escape

file2tinyurl
Purpose
This script will upload a file to TinyURL. Yeah, you heard that right.

Theory
TinyURL takes a given URL, and sticks it, with a pointer, into it’s database. Why does the URL have to be a URL? It could be any text. Well why not binary data? Just encode the binary data as text, like your mail client does every time you send a binary attachment, and upload that. This script does that for you, too. Then you just need a way to get it out again. See the next post.

What this script does

  1. When given a path to a file name, this will:
  2. Read the file
  3. UUencode the data in the file, converting it to “safe” ASCII.
  4. Strip the newlines from that, since UUencoded data is set at a certain width, and replace them with flags.
  5. Escape and non-URL safe text in the %xx form (ie, ‘~’ becomes %7E, ‘ ‘ becomes %20).
  6. POST the data to TinyURL, and strip the TinyURL and TinyURL ID from the results, displaying it to a console.

As of yet, I have not found a length limit on the original URLs that TinyURL will shorten. Therefore, as of yet, I have not found a filesize limit for “uploaded” files.

Source: file2tinyurl.pl

#!/usr/bin/perl

### This script was written by cmantito
### cmantito@cmantito.com
### http://cmantito.com
### This script can be freely modified/distributed/used and is open-source.
### However, I'd appreciate some credit on derivative works.

use LWP::UserAgent;
use Convert::UU qw(uudecode uuencode);
use HTML::Entities;
use URI::Escape;

## If this script runs with strict, you're fucked. So don't add strict.

$filename = shift(@ARGV);

if(!$filename){
   print "Usage: file2tinyurl /path/to/file\n";
   exit(15);
}

$relFn = $filename;
$relFn =~ s/^(.+)\///ig;

if((-e $filename) && (-r $filename)){
   if(-d $filename){
      fatal("Specified file must not be a directory.");
   }
}else{
   fatal("Specified file doesn't exist or isn't readable.");
}

open(TUFILE, "){
   $rawData .= $_;
}
close(TUFILE);

$uuData = uuencode($rawData);
$uuData =~ s/\n/_NWLN_/ig;

$urlData = uri_escape($uuData);

$url = "http://".$relFn."/".$urlData;
$formData{'url'} = $url;
$formUrl = "http://tinyurl.com/create.php";

$browser = LWP::UserAgent->new;
$response = $browser->post($formUrl, Content => \%formData);

if($response->is_success){
   $tinyData = $response->content;
   ($tinyId) = $tinyData =~ /\http:\/\/tinyurl\.com\/(.+?)\/;
   print "File uploaded successfully.\n";
   print "TinyURL ID (used with tinyurl2file for download): ".$tinyId."\n";
   print  "TinyURL (for reference): http://tinyurl.com/".$tinyId."\n";
   exit(0);
}else{
   print $formUrl;
   fatal("Couldn't contact TinyURL: ".$response->status_line);
}

sub fatal {
   $errorString = shift(@_);
   print STDERR "Fatal error: ".$errorString."\n";
   exit(5);
}

tinyurl2file
Purpose
This script will download a file that was uploaded to TinyURL with file2tinyurl.

Theory
TinyURL takes a given URL, and sticks it, with a pointer, into it’s database. Why does the URL have to be a URL? It could be any text. Well why not binary data? Just encode the binary data as text, like your mail client does every time you send a binary attachment, and upload that. Then you just need a way to get it out again. And this is that portion of it. ^_^

What this script does

  1. When given a TinyURL ID (the part of the TinyURL after the tinyurl.com/),
  2. Retrieves the preview page (preview.tinyurl.com/)
  3. Strips the “original URL” portion out of the page
  4. Extracts the original filename from our “URL”
  5. Looks for newline flags, and puts them back where they belong
  6. Gets rid of any HTML entities, replacing them with the correct ASCII (ie, < should be < and & should be &)
  7. Converts the URL-encoded characters back to ASCII (ie, %20 becomes a space, %7E becomes a ‘~’)
  8. Un-UUencodes the ASCII, getting it back to it’s original form (likely binary data)
  9. Writes that back out to a file, using the original file’s filename which was encoded into it with file2tinyurl.

Source: tinyurl2file.pl

#!/usr/bin/perl

### This script was written by cmantito
### cmantito@cmantito.com
### http://cmantito.com
### This script can be freely modified/distributed/used and is open-source.
### However, I'd appreciate some credit on derivative works.

use LWP::UserAgent;
use Convert::UU qw(uudecode uuencode);
use HTML::Entities;
use URI::Escape;

## If this script runs with strict, you're fucked. So don't add strict.

$urlId = shift(@ARGV);

if(!$urlId){
   print "Usage: tinyurl2file [TinyURL ID from file2tinyurl]\n";
   exit(15);
}

$tinyUrl = "http://preview.tinyurl.com/".$urlId;

$browser = LWP::UserAgent->new;
$response = $browser->get($tinyUrl);

if($response->is_success){
   $tinyData = $response->content;
}else{
   fatal("Couldn't contact TinyURL: ".$response->status_line);
}

($uuUrl) = $tinyData =~ /\
\(.+?)\
\<\/b\>\<\/blockquote\>/; $uuUrl =~ s/\
//ig; ($filename, $uuData) = $uuUrl =~ /http:\/\/(.+?)\/(.+)$/; $uuData =~ s/_NWLN_/\n/ig; $uuData = decode_entities($uuData); $rawData = uri_unescape($uuData); $rawData = uudecode($rawData); open(TUFILE, ">".$filename) or fatal($!); print TUFILE $rawData; close(TUFILE); print "File downloaded to ./".$filename."\n"; sub fatal { $errorString = shift(@_); print STDERR "Fatal error: ".$errorString."\n"; exit(5); }