Thursday, May 26, 2011

GitHub additions!

I've ported some of my old projects over to git and uploaded them to github.
A much better solution than hosting raw source files on my web server!


The projects that have been ported:
talklikewarren A twitter bot that posts things that sound like Warren Ellis.
fakemiddleman - A twitter bot that posts things that sound like The Middleman.
hottest100 - A python script that created a live music video channel out of The Triple J Hottest 100.
top1m A squid redirector that prevents clients from visiting sites outside of the Alexa top 1 million.
twitbot - An example of how one might use twitter as a channel for command and control of malware.



Saturday, May 14, 2011

py360 - A speed increase (mostly)

I've updated py360 (from the patch note):

Changed the Partition class from preprocessing the entire partition during its constructor, instead it now will resolve files and directories on demand and store the results for later. Basically trading precomputation for memoization. gamertags.py runs about 90x faster, report360.py runs about the same (since it touches every file) and mounting is about 100x faster. These improvements are at the cost of all first time reads being slightly slower but no wasted preprocessing is done.


What isn't mentioned is that this will also make it much more likely that corrupted partitions will mount (which was why I started looking at this change at all). Also, gamertags.py and report360.py have been changed to be compatible with the new changes. The main difference is that using partition.allfiles does not necessarily return all files but rather all the processed files, use partition.walk() to get all files. The old behaviour is still available by passing in precache=True to the Partition constructor. 

This is a fairly experimental patch so let me know if it doesn't work for you. Next on the dev list remains better output from report360.py (and STFS / XDBF).

I also hope to write some posts unrelated to py360 soon too!

Tuesday, May 10, 2011

py360 - Update!

Thanks to the feedback and test data provided by some excellent people (Thanks Juri, Matt and DC) I've managed to fix several bugs in py360 and it should now be a smoother experience.

The biggest fix is in the STFS parser which would naively assume that all filelisting blocks were contiguous, this fix means that STFS files with large numbers of files inside them will now work (e.g. Profiles that see a lot of use). Now report360.py should run cleanly on (hopefully) all images and the only errors you will see will be from trying to parse deleted files (they are recognisable by the ~ that precedes their name).

Next up on the dev list is to investigate the unicode parsing problems that occasionally appear and cause extraneous bytes (usually nulls) to appear in the output. Worst case I plan on changing report360.py to remove the null bytes, best case I find the underlying cause and sort that out.

If you would like to help with py360 I'd like to hear from you. I'm really interested in hearing about your experiences with py360 and especially tell me of any errors of inconsistencies that you encounter. If you're a programmer and really keen feel free to contribute code —especially example programs and bugfixes— I won't turn you away!

Saturday, May 7, 2011

py360 - An example to print out gamertags

EDIT: The code below is obsolete check github for an updated version.

To help people get started with py360 I written a simple example (simple at least compared to report360.py). gamertags.py takes an Xbox 360 disk image and prints out the gamertags of all the profiles present.

It produces output like:
Gamertag: Han Solo, Type: Gold (Paid)
Gamertag: Chewbacca, Type: Silver (Free)

The code is simple enough that I've included it below, it's also now in the git repository.

# An example of using py360 to extract all the GamerTags on a drive.

# Where does this data live?
# GamerTags are inside the Account block of a user STFS container
# located on the XTAF partition. 

# How do you find user STFS containers?
# Gamer profiles are located in the /Content directory in subdirectories
# named with 16 hex characters starting with an 'E' such as
# E00012DD5A4FAEE5. The STFS container is located in the 
# FFFE07D1/00010000 subdirectory and is named the same as the
# profile directory.
# Example: /Content/E00012DD5A4FAEE5/FFFE07D1/00010000/E00012DD5A4FAEE5 

from py360 import partition, stfs, account
import sys

# First, open the xbox 360 image
part = partition.Partition(sys.argv[1])

# Second, find profile STFS containers
for directory in part.allfiles['/Content'].files:
  if len(directory) == 16 and directory[0] == 'E':
    # Open each STFS container and look for the Account block
        
    # The STFS class can take either an actual file or a file-like object,
    # we're using an file-like object to avoid having to use a temp file.
    path = '/Content/%s/FFFE07D1/00010000/%s' % (directory, directory)

    # This test is to exclude deleted profiles and defunct directories
    if path in part.allfiles:
      profile = stfs.STFS(filename = None, fd = part.open_fd(path))

      # The account block is always at /Account in the STFS archive
      # we'll read it in, decode it and then print out the gamertag
      acc = account.Account(profile.read_file(profile.allfiles['/Account']))
      print "Gamertag: %s, Type: %s" % (acc.get_gamertag(), acc.live_type)