Aaron Winborn

As KarenS predicted, there have been many Drupal-free nights from me this summer, while my household adjusts to its newest member. Though thanks to Advomatic, my days have continued to be filled with Drupal.

For instance, I've been hard at work on Node Reference / Embed Media Browser (nrembrowser), the perhaps unfortunately named, but descriptive, module that does just what it says. I did a session for it in an earlier incarnation at Drupal Dojo, and have banged at it incessantly, until it's now just about ready for prime time! As I said in the session, that project is, I believe, a reasonable alternative for folks who want Media-like functionality for Drupal 6.

What makes it viable is its integration with WYSIWYG and Styles. Styles is a brainstorm I had during the development of Media, in which I tackled how to display wildly differing data (such as photos and videos) with a single field formatter. I've been refining the model in version 2 of the Drupal 6 version, where I've moved it into a class structure and made lots of improvements. Coming soon to Drupal 7...

And then there's Views Slideshow: Galleria, which makes it easy to create a Galleria slideshow in Drupal. And some refactoring of Embedded Media Field, version 3, in which we begin the migration to Drupal 7's Media, and bring some of its magic (of a unified storage system) back to 6...

But what I've been dying to tell you all is about my few, but precious, Drupal nights. Though it's gone from 12 hours or more a week to three, in the wee hours of the morning at that, I've still been banging away. However, I've changed directions entirely, following another passion of mine: games! And no, not playing them, making them... I've said too much already, as it's not quite ready for release. But suffice to say, I think it'll be fun, and of course, it'll be available through the good ol' GPL...

I'm actually posting this as a question. If you're looking for the answer, sorry I don't have it yet.

How can we reasonably handle large file uploads? I'm talking in the >100MB range; YouTube, for instance, now supports 2GB files, and this will become increasingly the norm. I don't think that most servers are up to that yet, particularly if you need an application to scale.

Currently, using PHP, you need to set memory_limit to more than twice the upload_max_filesize, which as you can see would be prohibitive in the example of 2GB uploads; you'd need to set your PHP memory to >4GB (adding the buffer of 64M or whatever you need to run Drupal). EDIT: Looks like I was incorrect in my assumption; if you're not going to process the file, you don't need a huge memory footprint just to handle the raw uploads. Thanks Nate and Jamie!

Even if you manage to have that kind of resource available, you can probably expect things to splode with concurrent uploads...

So I spent some time yesterday looking at SWFUpload yesterday (module here), as I'd misunderstood its claims. Yes, it handles large file uploads (from the browser's standpoint), but you still need to set PHP memory accordingly. Not suitable for what I'm looking for, but it is a really nice way to handle multiple uploads. WARNING: I also learned from experience and much head-scratching that it doesn't work if you have Apache authentication on your server...

Now I'm looking at node.js as a possibility. This looks really great, and might do the job. Basically, it's a JavaScript application that sits on your server. Yes, you heard that right. Turns out that as JS has evolved, it's turned into a really tight language, and should be quite suitable for concurrent tasks.

Sorry if you came to this post looking for answers; I've simply postulated more questions. But I'm hoping that someone with more experience with this issue might be able to comment, and we'll all benefit from it. Additionally, this might turn out to be a handy addition to the Media suite, perhaps as a fancy stream wrapper for handling large files? And I'll definitely follow-up when I figure out how best to tackle this.

Thanks,
Aaron

Thanks to some fine work by Jacob Singh, Dipen Chaudhary and myself over the past month, we've gotten Media working fairly well with WYSIWYG, with Media: YouTube and Media: Flickr to boot.

Click through to see the action...

If you haven't looked recently, there's been some huge progress recently for Drupal's Media module. Jacob Singh from Acquia has jumped on board, paving the way for fieldable entities! This allows Media asset objects to be a first class Drupal citizen, alongside Nodes, Users, Taxonomy, and Comments. (Hopefully in core for Drupal 8!) Also, Dipen Chaudhary has been hard at work providing WYSIWYG support!

Read more for a tutorial...

I'm adding display formatters to the Media module, and could use some feedback.

Basically, I'm taking the work from the Image Styles built into core (which is a port of Imagecache), and building a wrapper around it. A Media Style would be a collection of styles, based on the stream (public://, private://, youtube://, etc) and file mimetype (image/jpeg, image/png, etc.), that would be applied to a specific filefield display (either in the node teaser/page display, or a view field display, or possibly other places, such as inline).

As an example, you might have a 'small-box' Media style that contains a 'medium' image style, a 'preview' youtube style, and an 'inline' pdf style. Thus, if the filefield contained an image, it would display it with the image scaling, a youtube video with a small player, and a pdf would be displayed in an iframe. Undefined streams/mimetypes would fall back to the default file listing.

The module is intended to work stand-alone, with File, Image, and/or Media. Thus, one question I have is if it should be bundled with the Media module, or packaged outside the module. On the one hand, as it can be run w/o Media, it might be useful in other situations. On the other hand, I imagine 98% of Media users would also want this module, so I'm hesitant to create a new external dependency. I've mostly decided to bundle it with the Media module, but am open to new considerations I haven't had yet.

I've included two screen shots. The first screen shot at the top shows current functionality. Clicking on a radio will automatically load a new preview of the selected style (that part's not built yet, but that's the idea).

The second below was the original display for administration. It's now actually using vertical tabs, as with the first, but I'm including it since it gives a large overview (and is what it would look like w/o javascript anyway).

I'm mostly looking for feedback of how the administrative UI could be improved, as well as how to word instructions to the user that won't scare them off. Also, for any other conceptual, architectural, or other comments.

Thanks,
Aaron

(Cross-posted in the Drupal Media Group.)

If you haven't gotten your ticket for DrupalCamp Colorado, there's still time! Along with other excellent and anticipated sessions, I'll be presenting the current state of Drupal Media on Sunday, July 28, at 2:30 PM. The next day, Denver Open Media is hosting a Media Code Shiai / Sprint -- interested developers of all levels are invited and welcome!

For anyone interested, here's the latest mockup for the Media module, courtesy of Maarten Verbaarschot (mverbaar) of the D7UX team!

(original at http://www.flickr.com/photos/mverbaar/3632702590/in/set-72157619245872526/)

You can see more screenshots at the D7UX Microproject - Media Library for D7 issue.

And Jon Stacey (jmstacey) is doing a bang-up job of helping to realize this vision, via the Google Summer of Code! He's been hard at work with Andrew Morton (drewish), another co-mentor, to make this happen for Drupal 7. (And yes, we're also continuing to back-port this work into 6.)