libgpod is a library meant to abstract access to an iPod content. It
provides an easy to use API to retrieve the list of files and playlist
stored on an iPod, to modify them and to save them back to the iPod.

This code was originally part of gtkpod (www.gtkpod.org). When the iPod
content parsing code was made to be self-contained with gtkpod 0.93,
we chose to put this code in a separate library so that other project
can benefit from it without duplicating code.

Currently (2010-03-23) libgpod supports writing to all "classic" iPod models,
all iPod Nanos, iPod Minis and iPhones and iPod Touch. iPod Nano 5th Gen. and
iPhone/iPod Touch support is "partial" in the sense that a database with at
least one song written by iTunes is needed the first time you use the iPod
with libgpod. For these models, it's also highly recommended to get the
provided udev/hal callout to work for proper support. Older iPod Shuffle models
are supported, but the new button-less ones (3rd and 4th Gen.) are not. Please
get in touch if you want to hack on adding support for that.

For supported models, covert art and photos are supported in addition to
manipulating the music database.

If you decide to make improvements,  or have bug fixes to report or contribute,
just drop a mail to Gtkpod-devel@lists.sourceforge.net (due to too much spam,
the mailing list is unfortunately subscriber-only).

----------------------------------------------------------------------

Quick HOWTO use libgpod for audio

itdb_parse(): read the iTunesDB and ArtworkDB
itdb_write(): write the iTunesDB and ArtworkDB

itdb_parse() will return a Itdb_iTunesDB structure with GLists
containing all tracks (each track is represented by a Itdb_Track
structure) and the playlists (each playlist is represented by a
Itdb_Playlist structure).

A number of functions for adding, removing, duplicating tracks are
available. Please see itdb.h for details (itdb_track_*()).

In each Itdb_Playlist structure you can find a GList called 'members'
with listing all member tracks. Each track referenced in a playlist
must also be present in the tracks GList of the iTunesDB.

The iPod must contain one master playlist (MPL) containing all tracks
accessible on the iPod through the
Music->Tracks/Albums/Artists... menu. Besides the MPL there can be a
number of normal playlists accessible through the Music->Playlists
menu on the iPod. Tracks that are a member of one of these normal
playlists must also be a member of the MPL.

The Podcasts playlist is just another playlist with some internal
flags set differently. Also, member tracks in the Podcasts playlist
are not normally members of the MPL (so on the iPod they will only
show up under the Podcasts menu). All tracks referenced must be in the
tracklist of the Itdb_iTunesDB, however.

A number of functions to add/remove playlists, or add/remove tracks
are available. Please see itdb.h for details (itdb_playlist_*()).

Each track can have a thumbnail associated with it. You can retrieve a
GdkPixmap of the thumbnail using itdb_artwork_get_pixbuf().  You can
remove a thumbnail with itdb_track_remove_thumbnails(). And finally,
you can set a new thumbnail using itdb_track_set_thumbnails().

Please note that iTunes additionally stores the artwork as tags in the
original music file. That's also from where the data is read when
artwork is displayed in iTunes, and there can be more than one piece
of artwork. libgpod does not store the artwork as tags in the original
music file. As a consequence, if iTunes attempts to access the
artwork, it will find none, and remove libgpod's artwork. Luckily,
iTunes will only attempt to access the artwork if you select a track
in iTunes. (To work around this, gtkpod keeps a list of the original
filename of all artwork and silently adds the thumbnails if they were
'lost'. Your application might want to do something similar, or you
can supply patches for (optionally!) adding tags to the original music
files.)

The Itdb_iTunesDB, Itdb_Playlist and Itdb_Track structures each have a
userdata and a usertype field that can be used by the application to
store application-specific additional data. If userdata is a pointer
to an external structure, you can supply a ItdbUserDataDuplicateFunc
and a ItdbUserDataDestroyFunc so that this data can be duplicated
or freed automatically with a call to the library _duplicate()/_free()
functions.

For more information I would advice to have a look at gtkpod's source
code. You can also ask questions on the developer's mailing list:
gtkpod-devel at lists dot sourceforge dot net


Jörg Schuler (jcsjcs at users dot sourceforge dot net)

----------------------------------------------------------------------

Quick HOWTO use libgpod for photos

   itdb_photodb_parse():
       Read an existing PhotoDB.

   itdb_photodb_create():
       Create a new Itdb_PhotoDB structure. The Photo Library Album is
       (first album) is created automatically.

   itdb_photodb_add_photo(), itdb_photodb_add_photo_from_data():
       Add a photo to the PhotoDB (from file or from a chunk of
       memory). It is automatically added to the Photo Library Album
       (first album), which is created if it does not exist already.

   itdb_photodb_photoalbum_create():
       Create and add a new photoalbum.

   itdb_photodb_photoalbum_add_photo():
       Add a photo (Itdb_Artwork) to an existing photoalbum.

   itdb_photodb_photoalbum_remove():
       Remove an existing photoalbum. Pictures can be kept in the
       Photo Library or automatically removed as well.

   itdb_photodb_remove_photo():
       Remove a photo either from a photoalbum or completely from the database.

   itdb_photodb_write():
       Write out your PhotoDB.

   itdb_photodb_free():
       Free all memory taken by the PhotoDB.

   itdb_photodb_photoalbum_by_name():
       Find the first photoalbum with a given name or the Photo
       Library Album if called with no name.


If you cannot add photos because your iPod is not recognized, you may
have to set the iPod model by calling

itdb_device_set_sysinfo (db->device, "ModelNumStr", model);

For example, "MA450" would stand for an 80 GB 6th generation iPod
Video. See itdb_device.c for a list of supported models.

This information will be written to the iPod when the PhotoDB is saved
(itdb_device_write_sysinfo() is called).

Have a look at the following test-photos test program in the tests/
subdirectory for an example of how to use the interface.


Jörg Schuler (jcsjcs at users dot sourceforge dot net)

----------------------------------------------------------------------

Starting with the iPod Classics and the Video Nanos, libgpod needs an
additional configuration step to correctly modify the iPod content. libgpod
needs to know the so-called iPod "firewire id", otherwise the iPod won't
recognize what libgpod wrote to it and will behave as if it's empty.

There are several ways to set up an iPod so libgpod can find its firewire id.

The preferred method is automatic. Make sure you have hal and libsgutils
installed before running configure/autogen.sh. If you built libgpod without
them, run configure/make/make install after you install them.

A hal callout and .fdi file will be built and installed. This will query an
iPod when it is plugged in and save the SysInfoExtended file in the proper
place. This should be entirely automatic. If you have trouble with this, see
the TROUBLESHOOTING file for some hints.

If you build with libsgutils but without hal, the next best method is mostly
automatic. You should have an ipod-read-sysinfo-extended tool available. Run
it with the iPod device path and the iPod mount point /mnt/ipod) as arguments.
For example:

    $ ipod-read-sysinfo-extended /dev/sda /mnt/ipod

This may require root privileges. It reads an XML file from the iPod and
writes it as /mnt/ipod/iPod_Control/Device/SysInfoExtended. More details on
this method can be found at http://ipodlinux.org/Device_Information.

Having the SysInfoExtended file created by ipod-read-sysinfo-extended or the
hal callout is enough for libgpod to figure out the iPod firewire id.

The last method requires more manual intervention. First, you need to
determine the firewire id of the iPod. To do that on Linux, plug in the iPod
in and run (with root privileges):

    $ lsusb -v | grep -i Serial

This should print a 16 character long string like 00A1234567891231. For an
iPod Touch, this number will be much longer than 16 characters, the firewire
ID is constituted by the first 16 characters.

On FreeBSD, there is a tool to get the serial number at:

    http://50hz.ws/dev/getserial.c

Once you have the serial number, edit /mnt/ipod/iPod_Control/Device/SysInfo,
creating the file if it does not exist. (Replace /mnt/ipod with the path to
where the iPod is mounted). Add a line like this to the SysInfo file:

FirewireGuid: 0xffffffffffffffff

Replace ffffffffffffffff with the serial number you obtained in the previous
step. Don't forget the 0x before the string. After you add the FirewireGuid to
the SysInfo file you need to rewrite the iTunesDB for the change to take
effect. For example, add a new song or adjust the playcount of an existing
song and save the changes in gtkpod.

Be careful when using applications which let you manually specify the iPod
model. They may overwrite the SysInfo file and undo the changes.

Finally, if you compiled libgpod from source, you can test that libgpod can
find the firewire ID on the iPod using the test-firewire-id program in the
tests/ dir of the libgpod source. For example:

    $ cd ~/src/libgpod/tests
    $ ./test-firewire-id /ipod/mount/point

Introduction:

The goal of this document is to provide an overview of the various parts/files
involved when libgpod interacts with a device. Ideally it will be helpful to
help application writers/distributors/... figure out what's going wrong when
trying to interact with an iPod-like device.

Please note that not all devices need all these steps (especially the
hash58/hash72 stuff and the sqlite databases), see the table at the end for
more details


Overview:

* device is plugged in

HAL/udev detects the insertion and runs a callout. This callout sends the
appropriate commands (raw SCSI/USB commands or special AFC command) to the
device to get an XML file describing the device capabilities (artwork formats
supported, serial number, ...) and dumps this XML file to
iPod_Control/Device/SysInfoExtended for future use by libgpod. libgpod doesn't
do it directly because sending these SCSI/USB commands might need elevated
priviledges.

* the application uses libgpod to read the device content

iPod_Control/iTunes/iTunesDB is accessed as well as a few other files. If
the device needs an hash72 (either in its iTunesDB or because it's using sqlite
databases), then the necessary information is extracted from the existing
iTunesDB to generate an iPod_Control/Device/HashInfo file (if it doesn't
exist) which will be useful to generate any hash72 we need for this device.

* the application uses libgpod to write to the device

libgpod generates an iPod_Control/iTunes/iTunesDB file. If an hash58 is needed
then the device FirewireID is used to generate it. If an hash72 is needed
then the HashInfo file is used to generate it. If the device uses sqlite
databases, they are generated as well. Post process SQL commands are then
extracted from the device (either by getting them from SysInfoExtended or by
getting them through AFC) and executed after the sqlite database generation.
These post process commands are useful to make sure the sqlite databases layout
matches what the current device firmware expects.


Glossary:

iTunesDB: binary file containing information about all the songs, playlists, ...
stored on the device. On recent devices (iPhoneOS 3.x, Nano 5g), it's replaced
by iTunesCDB which is a compressed version of iTunesDB.

sqlite: recent devices (iPhoneOS 3.x, Nano5g) use sqlite databases instead
of an iTunesDB to store song information, playlists, ... More accurately, they
have both an iTunesCDB (compressed iTunesDB) used by iTunes to know the device
content and sqlite databases used by the device. Along with these sqlite
databases, there's a .cbk file which contains checksum information and a hash72
for the sqlite data.

iPhoneOS: umbrella for iPhone-like devices, ie iPhones, iPod Touch and (likely)
iPad. On these devices, what matters is the firmware version, not the device
type.

hash58: first iTunesDB hashing scheme introduced by Apple. It appeared in the
iPod Nano Video (3g)  and iPod Classic and was used by iPhoneOS 1.x. It's fully
reverse-engineered and uses the iPod FirewireID (called this way even on USB
devices) as part of the calculation.

hash72: hashing scheme that was first introduced in iPhoneOS 2.x and has then
been used on the iPod Nano with a camera (5g). It was much more complicated to
reverse engineer and is not yet 100% known. However, given a file with a valid
hash72, we can extract some hashing data to be able to generate valid hash72
hashes for any file.



iPod feature matrix:

		SysInfoExtended   hash58   hash72   sqlite   iTunesCDB
iPod 1G	                     no       no       no       no          no
iPod 2G	                     no       no       no       no          no
iPod 3G	                     no       no       no       no          no
Mini 1G	                     no       no       no       no          no
Mini 2G	                     no       no       no       no          no

iPod 4G                     yes       no       no       no          no
iPod 5G                     yes       no       no       no          no
Nano 1G                     yes       no       no       no          no
Nano 2G                     yes       no       no       no          no

iPod Classic                yes      yes       no       no          no
Nano3G                      yes      yes       no       no          no
Nano4G                      yes      yes       no       no          no
iPhoneOS 1.x                yes      yes       no       no          no

iPhoneOS 2.x                yes       no      yes       no          no

Nano5G                      yes   yes[1]      yes      yes         yes
iPhoneOS 3.x                yes       no      yes      yes         yes


[1] surprisingly, the Nano5G uses a hash58 for its iTunesCDB and a hash72
for its sqlite cbk file. iPhoneOS 3.x uses hash72 everywhere.

* Summary

For things to work with an iPod Nano 5G or an iPhone/iTouch, you need
  - an iPod_Control/Device/SysInfoExtended file (Nano5G) or
  - an iTunes_Control/Device/SysInfoExtended file
This file should be created when the device is plugged in. If not, you can
generate it manually using ipod-read-sysinfo-extended (passing the device UUID
as an argument for an iPhone/iPod Touch and the USB bus and device number for
a Nano 5G). On a Nano5G, passing the device node (/dev/sdb) to
ipod-read-sysinfo-extended won't work. On a Nano5G, SysInfoExtended should be
a few dozen kilobytes in size.

You also need a database generated by iTunes on your iPod when you first plug
it in. When libgpod reads this database, it will generate
  - an iPod_Control/Device/HashInfo file (Nano5G) or
  - an iTunes_Control/Device/HashInfo file (iPhone/iPod Touch)
which is required to write to the device.



* sqlite

New iPod models (iPhone/iPod Touch on the one hand, Nano 5G on the other hand)
contain both a compressed version of the usual iTunesDB (the iTunesCDB file)
and sqlite databases. The iTunesCDB file is used by iTunes while the sqlite
databases are used by the device. The sqlite databases use the .itdb extension
and can be found in iPod_Control/iTunes/iTunes Library/
(iTunes_Control/iTunes/iTunes Library/ on devices using the iPhone OS). Both
use some kind of checksum.

The iTunesCDB file is checksummed either with a hash58 (old hash introduced
with the first iPod Classic and the Nano 3g) or a hash72 (new hash introduced
with iPhone OS 2.x). The hash58 is used on the Nano 5G while the hash72 is
used on the iPhone/iPod Touch.

The sqlite databases are checksummed through the use of a .cbk file. This file
only checksums the content of the Location.itdb sqlite database and uses the
hash72 on the iPhone/iPod Touch and the Nano 5G.


* HashInfo

The HashInfo file can be found in iPod_Control/Device/HashInfo
(iTunes_Control/Device/HashInfo for an iPhone/iPod Touch). At the moment,
it's not known how to generate a checksum for the iPod database from scratch.
However, for a given iPod, we can extract the necessary data from a database
generated by iTunes to be able to generate a checksum for any other database
for this iPod.
libgpod extracts this data the first time it reads a database on an iPod (so
it assumes this database was generated by iTunes) and dumps it to the HashInfo
file (very simple format, see the struct Hash78Info in src/itdb_hash72.c).
Then libgpod can reuse the information from that file to write valid databases
for the iPod the HashInfo was generated on.


* SysInfoExtended

The SysInfoExtended file is stored in iPod_Control/Device/SysInfoExtended
(iTunes_Control/Device/SysInfoExtended on iPhone/iPod Touch). This file isn't
a standard iPod file, it's generated using some data available on the iPod
and dumped there because it's more convenient for libgpod. It's an XML file
containing lots of data about the iPod (serial number, support image and video
formats, ...). libgpod installs a HAL/udev callout to automatically generate
this file when an iPod is plugged in, but it can also be done manually by
using the ipod-read-sysinfo-extended tool.
This file is required on the Nano 5G because it contains SQL commands that
have to be run to generate sqlite databases that the device will be able to
handle.
The iPhone/iPod Touch need similar SQL commands to be run, but they are not
stored in SysInfoExtended, they are extracted from the device when needed.


On the iPhone/iPod Touch, the data for the SysInfoExtended file can be
obtained by asking for the right lockdown key through the usbmux daemon.
On the Nano5G, things are a bit more tricky. Older iPods models used to
make that data available through a SCSI inquiry command. This command works
on the Nano5G too, but it returns really partial data (no SQL post-process
commands, no artwork format information, ...). The full data can be obtained
through an USB Control command (see tools/ipod-usb.c for more details).