My Queue-Based Life

"In the future, there will be so much going on that no one will be able to keep track of it."

--David Byrne, "In the Future", from The Knee Plays

I love Instapaper. It helped solve the "20 browser tabs open" problem where you click on something interesting to read - but it turns out to be kinda long, and you don't have time to read it right now. In fact, you don't really want to read it on your work computer at all; you'd rather pull it up on your tablet in your living room with a fire going in the fireplace and a glass of Dogfish Head 60-Minute IPA at your side. Consequently, I tend to have a fairly long list of interesting articles queued up to read at any given time, lovely articles from The Atlantic and Slate and The New Yorker and Vanity Fair and Wired and Rolling Stone and qz.com and the like.

However, the Instapaper articles compete with digital issues of Newsweek, which are full of articles about NEWS which come out EVERY SINGLE WEEK. I'm usually about 6-8 issues behind, which means that I can sometimes skip articles because the Big Question they're asking has been answered, or is no longer relevant at all.

Newsweek and Instapaper, of course, compete for my reading time with books, still mostly paper but some digital. Authors have this annoying habit of writing books on fascinating subjects that I can't wait to dive into, and so those titles are added to the list I keep on a Google Drive spreadsheet, which is also more or less duplicated in a Goodreads account. Many of these books have been purchased and sit there, waiting to be read as soon as I can get around to them.

If I don't feel like reading, no problem - TiVo dutifully records all kinds of interesting TV programs, hours and hours and hours worth, more than my wife and I could ever hope to watch. Lately, an "AFI's List of 100 Greatest Movies" smart search thingy has intersected with an enhanced cable subscription that gives us Turner Classic Movies, and the TiVo lit up like a pinball machine, recording "Casablanca" and "Citizen Kane" and "The African Queen" and "Bringing Up Baby" and "Platoon" and "The Last Picture Show" and so on and so on.

If we're not in the mood for something on TiVo, Netflix has us covered - there are maybe 50 movies and a few TV series in the queue there. Who knows if we'll ever get around to watching them? Actually, I know - we'll never get around to watching them all. And that's not including the dozens of TV shows or movies that my friends tell me I MUST watch - how could I be missing out on Game of Thrones or Fargo or The Walking Dead or Adventure Time or Girls or Black-ish or Doctor Who or The Knick, etc, etc, etc.

Meanwhile, when I'm exercising or driving or doing mundane household chores or just lying sick in bed, there are the podcasts. I've discovered I'm something of a podcast junkie. It is so, so easy to subscribe to a new one, promising hours and hours of interesting listening - techie podcasts like Hanselminutes and .NET Rocks; wonky podcasts like Planet Money and Freakonomics and The Commonwealth Club of California; the Slate Political and Culture gabfests; Marc Maron's WTF interviews, and just damned interesting podcasts like Radiolab and 99% Invisible. I'm months behind on some of those.

This may be the ultimate First World Problem. There is so much quality content being produced, by so many amazingly talented writers/thinkers/researchers/producers/directors/actors/singers/playwrights/I/could/go/on/and/on that I occasionally stress out that I'm not consuming it fast enough. Each separate stream of content threatens to become its own to-do list to be worked through as quickly and efficiently as possible, like Lucy and Ethel wrapping candies at the conveyor belt.

I have, sometimes, learned to let go, not worry about all of it piling up, and just cherry-pick the best stuff .This doesn't solve the larger problem, however, which is that consuming all this stuff - as great as it can be - is not the same as creating something or doing something. Yes, I am an info-junkie, but i have to remember to put that information to good use in the service of action, not merely learning for learning's sake, as attractive as that can be.

Creating a Shuffled Classical Music Playlist

I have a reasonably large classical music collection, and sometimes I'm in the mood to put classical music on, but I don't want to choose exactly what. Yes, I could find some classical music radio station, either over the air or on the internet, but having grown up in an age before universal streaming, sometimes I just want to listen to my music. I'm generally a big fan of randomly shuffled music, and I like Smart Playlists in iTunes. Using smart playlists, I can set up metadata-based playlists like "play a random selection of all rock or blues tracks rated 4 stars or higher than have not been played in the last 3 months."

Classical music is different than rock, pop, blues, jazz, country or most other forms of music. Besides the obvious things, you're often more interested in the composer rather than the performer. For example, I know I like Beethoven better than Vivaldi, but I wouldn't say that I like the Boston Symphony better than the New York Philharmonic. (This isn't 100% true, particularly for solo artists - I might have particular pianists, violinists, etc. that I really like.)

You'll notice another difference with classical music if you try to listen to it by shuffling tracks. You might get the second movement of a Mendelssohn symphony, followed by the third movement of a string quartet, followed by the first movement of a piano concerto, followed by the William Tell Overture. This is not quite what I'm looking for - I'd rather have it play the entire symphony, followed by the entire string quartet, followed by the whole piano concerto, followed by the William Tell Overture.

I could shuffle by album, but that's not quite "shuffle-y" enough. Plus, occasionally I have a piece that spans two discs. I have a nice recording of Tchaikovsky's Symphonies 4, 5, and 6 on two discs, but the poor Fifth Symphony gets split across the discs.

In other words, I don't want to shuffle by track but by work. A classical work is the typical unit of composition and performance - Beethoven wrote the 5th symphony as a cohesive whole, and usually all four movements are performed together and in succession.

Fortunately, iTunes supports classical composers and works, sort of. If you right-click on a track and choose "Get Info", you'll see something like this:

There is a specific field for Composer, but what about Work? Well, it turns out that iTunes treats the Grouping field as the work, at least for tracks in the Classical genre. You used to be able to see this in the Classical Music smart playlist in older version of iTunes, but apparently they've changed things around in more recent versions so it doesn't say anything about the work there anymore.

To aid in the effort of adding composer and grouping information, I made smart playlists called "Classical with No Composer" and "Classical with No Work Grouping", which help to identify where I need to add metadata:

It's relatively easy to add composer information to everything, but quite a bit more work to add Work groupings if you have a large classical music collection. It turns out that having the Work entered for everything is nice, but not critically important, as I'll get to in a minute.

What I want, then, is to get all the tracks in the Classical genre that have a composer, group them by work, then shuffle the works, while preserving the track order within the work. Building on some earlier work coding against the iTunes API, this ultimately evolved into a high-level LINQ expression that's fairly expressive:

                 List<IITFileOrCDTrack > tracks =

                    iTunes.Library.FileTracks

                        .InGenre( "Classical")

                        .WithComposer()

                        .WithFile()

                        .ShuffleByWork()

                        .ToList();

  

This expression uses several extension methods on IEnumerable<IITFileOrCDTrack>:

         public static IEnumerable< IITFileOrCDTrack> InGenre(this IEnumerable<IITFileOrCDTrack > trackCollection, string genreName)

        {

            return from IITFileOrCDTrack track in trackCollection

                where track.Genre == genreName

                select track;

        }

        public static IEnumerable< IITFileOrCDTrack> WithComposer(this IEnumerable<IITFileOrCDTrack > trackCollection)

        {

            return from IITFileOrCDTrack track in trackCollection

                   where !string .IsNullOrWhiteSpace(track.Composer)

                   select track;

        }

        public static IEnumerable< IITFileOrCDTrack> WithFile(this IEnumerable<IITFileOrCDTrack > trackCollection)

        {

            return from IITFileOrCDTrack track in trackCollection

                   where File .Exists(track.Location)

                   select track;

        }

  

These are pretty straightforward, but obviously the ShuffleByWork method is the hard part. But first, we need a simple ClassicalWork class:

    public class ClassicalWork

    {

        public string Composer { get; private set ; }

        public string Album { get; private set ; }

        public string Name { get; private set ; }

    }

  

This class also has a constructor, plus the System.Object overrides Equals(), GetHashCode(), and ToString() methods, but nothing spectacular here.

The ShuffleByWork() method starts out by grouping all the tracks into ClassicalWork objects:

             var workGroups = from filetrack in filetracks

                             group filetrack by new ClassicalWork(filetrack.Composer, filetrack.Album, Classical.WorkName(filetrack.Grouping, filetrack.Name));

  

The Classical.WorkName() static method constructs the best possible name for the work. If we have a Grouping for the particular track, well, we know that's the work name, so we go with that. If not, we look at the track name. Overtures are normally only one track, and are good to intersperse between longer works, so if the track name contains "overture" return it as the work name. Otherwise, we just an empty string, which means that this track isn't really part of a work (but we'll still include it in the shuffled playlist).

Grouping tracks in this manner is really what defines what we consider to be a work. All tracks on a particular album that have the same composer and work name (as defined above) are considered to be the same work as far as our shuffling goes. This means that all tracks on an album that have the same composer but don't have the same Grouping will still get grouped together into one pseudo-work. In practice, this is fine - if I have several tracks on an album by Sibelius, say, but don't know anything else about whether they fit together, it's perfectly OK to group them together.

So now workGroups is a list of all the works we're dealing with, each with a list of the tracks in the work. Next, we just shuffle all the work groups:

             var shuffledWorkGroups = workGroups.Shuffle();

  

The Shuffle() extension method is perfectly generic (in both senses of the word) and could be used to shuffle any collection - a deck of cards, etc:

        public static IEnumerable<T> Shuffle<T>( this IEnumerable <T> enumerable)

        {

            var random = new Random();

            var shuffled = from item in enumerable

                           orderby random.Next()

                           select item;

            return shuffled;

        }

  

The last thing to do in the ShuffleByWork() method is to iterate through each item in our shuffled work groupings, and return the tracks in each grouping ordered by the original track number:

             foreach (var workGroup in shuffledWorkGroups)

            {

                // Work name is workGroup.Key;

                var tracks = workGroup.OrderBy(t => t.TrackNumber);

                 foreach (IITFileOrCDTrack track in tracks)

                {

                    yield return track;

                }

            }

  

Next, we'll create a playlist, deleting an existing playlist if necessary:

                IITUserPlaylist playlist = iTunes.FindPlaylistByName(playlistName);

                if (playlist != null )

                {

                    playlist.Delete();

                }

                playlist = iTunes.CreatePlaylist(playlistName);

                foreach (var track in tracks)

                {

                    playlist.AddTrack(track);

                }

Once the program runs, the playlist obligingly shows up in iTunes:

Finally, we'll create an M3U playlist, which is a very simple text file and will allow non-iTunes apps to access the data. 

We'd like to shuffle this on a regular basis, so I need to create a little batch file that can be run as a scheduled task every night. The essence of that is this line:

ClassicalPlaylist.exe "Classical Shuffle" "%USERPROFILE%\Music\My Playlists\Classical Shuffle.m3u"

With that done, I can point all the various music devices and services in my home (Sonos, Plex, etc.) to that .m3u file,

The project and source code is up on Github if you want to take a look.