Browsing articles tagged with " tinyurl"

tinyurlfs, file2tinyurl & tinyurl2file

IRC Log (or, initial idea conception)

<Ignite> :)
<cmantito> ok, this is gonna sound crazy
<cmantito> really, really crazy
<cmantito> as you might have (or not) seen when it was in the /t, I wrote a set of scripts that could encode files
as URLs to be stored in a tinyurl, and then extracted again
<LeaChim> yes, that was quite cool
<cmantito> now, I'm writing a mountable filesystem, when you copy a file into it, it uploads it to tinyurl, grabs
the tinyurlid, and passes it to a PHP API running on my server and it stores the filename and the tinyurl ID
<LeaChim> why a php script?
<LeaChim> why can't it store this locally?
<cmantito> I'm getting there ;)
<cmantito> and when you read a file, it queries the API running on my server with the filename, gets the urlID, and
allows you to open the file in place
<cmantito> ideally, it's a write-once filesystem, you can put files into it, but not remove them, overwrite them, or
modify them.
<cmantito> the reason for using a remote API for it, is so that it's a unified filesystem
<cmantito> ie, I copy a file into it, and you can copy it right back out again
<cmantito> it's a giant filestore!
<LeaChim> heh
<cmantito> for *everyone*
<LeaChim> quite cool
<cmantito> and
<cmantito> if someone wants to bitch at me for "storing" files of naughty materials
<cmantito> I can use the bittorrent defense
<cmantito> "I only store metadata"
<Ignite> rofl
<LeaChim> heh
<LeaChim> well, you'll be implemented this as fuse right?
<cmantito> yes.
<cmantito> it's half-done
<Ignite> cmantito, so like
<Ignite> cmantito, we can all use it?
<cmantito> yes :D
<Ignite> So we can sort of share warez and shits?
<Ignite> :x
<cmantito> if the links are already in the database, it can read them and list them
<LeaChim> cmantito, for rmdir unlink rename chmod chown truncate you should just return permission denied
<cmantito> LeaChim: exactly! :D
<cmantito> fserr(13)
<cmantito> ENOACCES
<LeaChim> so all you've got left are mkdir symlink link utime, and file creation
<cmantito> well those 4 are unnecessary
<LeaChim> in fact, you might as well not have symlink or link
<LeaChim> no, keep mkdir
<cmantito> exactly
<cmantito> no, it's a flat filesystem
<Ignite> No
<LeaChim> boring
<Ignite> mkdir
<LeaChim> and horrible to list
<Ignite> WE WANT MKDIR
<LeaChim> add mkdir
* Ignite chants
<cmantito> ffs *fine*
<LeaChim> besides, mkdir is just php side
<Ignite> Rofl
<cmantito> now it's gonne take even longer.
<Ignite> xD
<cmantito> I *was* trying to keep it relatively simple
<cmantito> since if it takes off, tinyurl will kinda notice
<cmantito> Upload a DVD image "What's that, a 4.93921239x10^9 character URL!?"
<cmantito> actaully more
<cmantito> uuencode pads.
<cmantito> butokfine
<LeaChim> cmantito, you might want to add some code to allow you to split up files
<cmantito> I'll put in the ability to add one level of directories.
<cmantito> ONE XD
<LeaChim> lol
<LeaChim> why not more than one?
<LeaChim> it's just going to get horrible to list directories ;
<LeaChim> and maybe you should add something to expire things, so the directory listing doesn't end up being
gigantic
<cmantito> I'm trying to limit the amount of HTTP queries I have to make for any given requests.
<cmantito> s/(request)s/\1/

Original Scripts

The original file2tinyurl and tinyurl2file scripts were originally published at RackLoad (now defunct) and then at the Xelix wiki, followed by my own CodeWiki to continue development into a FUSE mountable filesystem. They now live here, source available below.

Development Roadmap

It actually works. By using file2tinyurl, and then putting the relevant values into the database manually. the files are accessible. It is not directory-structured yet, and file upload isn’t complete yet either. Please keep adding feature requests, and as soon as a usable version (even without ALL the features) is alpha, I’ll chuck it up here.

Original Proof of Concept Source

These scripts requires the following Perl modules:

  • LWP::UserAgent
  • Convert::UU
  • HTML::Entities
  • URI::Escape

These modules all have dependencies of their own. To easily install the modules and all their dependencies, execute the following command as root and it will prompt you to install the dependencies. Pressing return repeatedly will generally suffice.

cpan install LWP::UserAgent Convert::UU HTML::Entities URI::Escape

file2tinyurl
Purpose
This script will upload a file to TinyURL. Yeah, you heard that right.

Theory
TinyURL takes a given URL, and sticks it, with a pointer, into it’s database. Why does the URL have to be a URL? It could be any text. Well why not binary data? Just encode the binary data as text, like your mail client does every time you send a binary attachment, and upload that. This script does that for you, too. Then you just need a way to get it out again. See the next post.

What this script does

  1. When given a path to a file name, this will:
  2. Read the file
  3. UUencode the data in the file, converting it to “safe” ASCII.
  4. Strip the newlines from that, since UUencoded data is set at a certain width, and replace them with flags.
  5. Escape and non-URL safe text in the %xx form (ie, ‘~’ becomes %7E, ‘ ‘ becomes %20).
  6. POST the data to TinyURL, and strip the TinyURL and TinyURL ID from the results, displaying it to a console.

As of yet, I have not found a length limit on the original URLs that TinyURL will shorten. Therefore, as of yet, I have not found a filesize limit for “uploaded” files.

Source: file2tinyurl.pl

#!/usr/bin/perl

### This script was written by cmantito
### cmantito@cmantito.com
### http://cmantito.com
### This script can be freely modified/distributed/used and is open-source.
### However, I'd appreciate some credit on derivative works.

use LWP::UserAgent;
use Convert::UU qw(uudecode uuencode);
use HTML::Entities;
use URI::Escape;

## If this script runs with strict, you're fucked. So don't add strict.

$filename = shift(@ARGV);

if(!$filename){
   print "Usage: file2tinyurl /path/to/file\n";
   exit(15);
}

$relFn = $filename;
$relFn =~ s/^(.+)\///ig;

if((-e $filename) && (-r $filename)){
   if(-d $filename){
      fatal("Specified file must not be a directory.");
   }
}else{
   fatal("Specified file doesn't exist or isn't readable.");
}

open(TUFILE, "){
   $rawData .= $_;
}
close(TUFILE);

$uuData = uuencode($rawData);
$uuData =~ s/\n/_NWLN_/ig;

$urlData = uri_escape($uuData);

$url = "http://".$relFn."/".$urlData;
$formData{'url'} = $url;
$formUrl = "http://tinyurl.com/create.php";

$browser = LWP::UserAgent->new;
$response = $browser->post($formUrl, Content => \%formData);

if($response->is_success){
   $tinyData = $response->content;
   ($tinyId) = $tinyData =~ /\http:\/\/tinyurl\.com\/(.+?)\/;
   print "File uploaded successfully.\n";
   print "TinyURL ID (used with tinyurl2file for download): ".$tinyId."\n";
   print  "TinyURL (for reference): http://tinyurl.com/".$tinyId."\n";
   exit(0);
}else{
   print $formUrl;
   fatal("Couldn't contact TinyURL: ".$response->status_line);
}

sub fatal {
   $errorString = shift(@_);
   print STDERR "Fatal error: ".$errorString."\n";
   exit(5);
}

tinyurl2file
Purpose
This script will download a file that was uploaded to TinyURL with file2tinyurl.

Theory
TinyURL takes a given URL, and sticks it, with a pointer, into it’s database. Why does the URL have to be a URL? It could be any text. Well why not binary data? Just encode the binary data as text, like your mail client does every time you send a binary attachment, and upload that. Then you just need a way to get it out again. And this is that portion of it. ^_^

What this script does

  1. When given a TinyURL ID (the part of the TinyURL after the tinyurl.com/),
  2. Retrieves the preview page (preview.tinyurl.com/)
  3. Strips the “original URL” portion out of the page
  4. Extracts the original filename from our “URL”
  5. Looks for newline flags, and puts them back where they belong
  6. Gets rid of any HTML entities, replacing them with the correct ASCII (ie, < should be < and & should be &)
  7. Converts the URL-encoded characters back to ASCII (ie, %20 becomes a space, %7E becomes a ‘~’)
  8. Un-UUencodes the ASCII, getting it back to it’s original form (likely binary data)
  9. Writes that back out to a file, using the original file’s filename which was encoded into it with file2tinyurl.

Source: tinyurl2file.pl

#!/usr/bin/perl

### This script was written by cmantito
### cmantito@cmantito.com
### http://cmantito.com
### This script can be freely modified/distributed/used and is open-source.
### However, I'd appreciate some credit on derivative works.

use LWP::UserAgent;
use Convert::UU qw(uudecode uuencode);
use HTML::Entities;
use URI::Escape;

## If this script runs with strict, you're fucked. So don't add strict.

$urlId = shift(@ARGV);

if(!$urlId){
   print "Usage: tinyurl2file [TinyURL ID from file2tinyurl]\n";
   exit(15);
}

$tinyUrl = "http://preview.tinyurl.com/".$urlId;

$browser = LWP::UserAgent->new;
$response = $browser->get($tinyUrl);

if($response->is_success){
   $tinyData = $response->content;
}else{
   fatal("Couldn't contact TinyURL: ".$response->status_line);
}

($uuUrl) = $tinyData =~ /\
\(.+?)\
\<\/b\>\<\/blockquote\>/; $uuUrl =~ s/\
//ig; ($filename, $uuData) = $uuUrl =~ /http:\/\/(.+?)\/(.+)$/; $uuData =~ s/_NWLN_/\n/ig; $uuData = decode_entities($uuData); $rawData = uri_unescape($uuData); $rawData = uudecode($rawData); open(TUFILE, ">".$filename) or fatal($!); print TUFILE $rawData; close(TUFILE); print "File downloaded to ./".$filename."\n"; sub fatal { $errorString = shift(@_); print STDERR "Fatal error: ".$errorString."\n"; exit(5); }