We thought me might share thoughts around our file organization. Of course there are plenty of ways of organizing your work, and this is one of them. But it’s simple and complete enough to fit many animation projects. Before describing it we will start with generic tips and thoughts.
Origins
I must admit, the main organization of our files comes from my experience at Mac Guff animation studio. Back in 2009 my first job there was to clean Despicable Me and delete as many files as possible every week to save space for the project (around 2000 Gb per week). I had to be close to the file organization and I learned a lot!
It was not hard to understand the benefits of a good file organization. So when I left Illumination Mac Guff in 2013, I had organized several projects, either as technical director or supervisor. I was not going to fully reproduce what I have dealt with at Illumination, as it was too big, but I could make a simple version of it.
There are many ways to handle this question. If you are familiar with simple file naming, you might just skip the first paragraphs. As for seeing other ways of organizing your work, I can recommend reading Frank Rousseau’s post A proposal for your file hierarchy. It’s based on structures that I have witnessed in other studios too.
Libs and Films
When you deal with an animation movie there are two main types of files. The first one is assets. It could be characters, props or sets, but also pre-designed visual effects (vfx). In a nutshell, assets are individual elements used one or more times in a shot. Assets depend on diverse tasks to be completed. For example, a character goes from the design to the rig through modeling, sculpting, shading, rigging.
The second type of files are related to shots. They are stored within sequences within a movie. There is also a diverse list of tasks to complete a shot like Layout, Animation, Lighting, Compositing.
The assets are saved in a Library (as a library of assets), and the shots in a Film. So the two main categories are a LIB (Library) and a FILM.
Diversity
As we have seen, in both cases we have a diverse variety of tasks and departments involved. The type of files, the size, the work complexity, the software involved, can be different within the same project and from one department to another. Some assets are pretty simple and some are very complex. And sometimes it is only one person dealing with all of it, more often there are specialized people involved. Some software require just one file, other software might need a complex setup for each version, with several folders and different kinds of files for one department. And if you are working on rendering and compositing, your output is often an image sequence. A lot of individual images representing each frame.
So a huge diversity of parameters are to be handled in your project. And every project is unique. A new project might need to include new types of files. And during a production you could also need to include something new. So you’d better have a folder tree and naming convention ready to handle unexpected directions.
NB: when I use the term naming conventions in the rest of the article, I will refer to both directory naming and structure, and file naming.
The Generic Case
When designing your naming conventions, you should always think of it as a generic case. It must be something designed to handle most scenarios. No matter if you are dealing with a 3d file or a 2d compositing file, their naming should follow the same rules. The more generic you are, the more you will be able to handle most cases. But also, within teamwork, the easier it will be to navigate from a department to another if the file structure is the same.
You might need specific cases to be handled in a different way. But try to make it an exception. A real one: avoid making a collection of exceptions where no files are following the generic structure anymore.
And if the structure is consistent across a project, it’s way easier to script tools which are going through the folders and files.
You might decide to go from the largest case to the more atomic details of your production. That’s your hierarchy of folders and files, and orders the way you organize and save your data.
It’s Not Made for Humans
A file structure should not be made for humans. It’s not something you should define as a human task to build the folder hierarchy and navigate within it. If your naming is well structured you will quickly create tools that create all the folders, that open the files thanks to wizards within your software without having to search the file you are looking for.
Avoid using numbers to prefix
Something to avoid is prefixing your folders with numbers. It’s something I’ve seen a lot, to help people go through the folders. For example, numbering the departments of a shot:
- 1_Layout
- 2_Animation
- 4_Animation-Crowd
- 5_Lighting
- 6_Compositing
But are you going to create all those folders for every shot? What if you don’t have Animation Crowd? And above all, what if in the middle of the production some shots need a new department? For example, VFX simulations? Do you add it at the end? As in: 7_Vfx when it should be before lighting? Or worse, do you change the numbering? 5_Vfx, 6_Lighting, 7_Compositing?
This is going to be very hard to maintain through time and projects. I do not recommend to use numbers.
And if you are programming an asset manager, and create an object-oriented API, this will not work: Shot.1_Layout; as a python object is not allowed to be named starting with a number. So you might be stuck later programming a strong API.
Long names can be useful
Also, long names are fine too. We’ll get into that later, but it’s better to have: DIL_S01_P0001_Animation-Crowd_Render_v03.0101.exr than render.0101.exr when you open an image sequence folder. What the heck is the second one?
A file name should provide all the necessary information to identify the file without opening it. And, again, it might be painful at first, but you quickly have tools to organize the file names.
No special characters
Special characters (éöàñ or any other non-ascii character) are a nightmare to handle in scripts. It breaks everything! One day, when truly everything is Unicode, we might be safe, but until then: stay with the classics A to Z, “_” (underscore) or “-” (dash).
Also avoid spaces at any cost! It is also painful to handle with scripts. If you are defining a naming convention and don’t know about scripting and why I’m saying that, please, on behalf of all the programmers out there: trust me.
On upercase
Well, you should be carreful with upercase (ABC instead of abc) as it might have an impact on your file names, and some file systems allows you to have MyFile next to myfile. It might be confusing so you could either set a specific policy: For example, and as you will see bellow, we use them in the FILM codename, the shot and sequence previfx or the asset name (Camelcase style). Or you might decide something else : no upercase at all. So you will never have to remember when to use them or when not to.
Good luck with Windows
If you are on Windows, you might have noticed paths are represented with “\” (backslash); which is also something super annoying as that character is often the escape symbol and used in many languages. In python, if you write a Windows path it will be written like this: “C:\my\path\to\somewhere”. Because you must escape the escape character… And if you have a network path, look at that amazing four backslashes: “\\\\server_host\\path\\to\\somewhere”. I think it’s pretty ridiculous. But most libraries, like os, handle pretty well paths with usual slashs “/” as we find on unix systems or webpage urls. So you can do the following: os.path.exists(“C:\\path\\to\\myFile.blend”) or os.path.exists(“C:/path/to/myFile.blend”).
By the way, if you remember the paragraph about long names, you are limited to 255 characters in paths on most windows file systems. Lol.
In any case, good luck with Windows.
Padding
Always use padding when dealing with numbers. Padding means adding zeros even if you don’t need to. Usually when you talk about a padding of 4, it means a number composed of 4 digits. If you are representing 23, you write it as 0023.
It’s important for some scripts (ffmeg or some compositing tools for example) to handle your file sequences in an easier way. But it’s also because of a very stupid ordering thing. Some operating systems or languages, when asked for a list of files, might order them like this:
- render.1.exr
- render.10.exr
- render.11.exr
- render.2.exr
- render.3.exr
- …
- render.9.exr
It’s because it’s following a strict alphabetical sorting. It could be very annoying if your software does not read the files in the proper order, right?
That does not happen if you are using padding (here a padding (of ?) 3: 3 digit used to represent the number):
- render.001.exr
- render.002.exr
- render.003.exr
- …
- render.009.exr
- render.010.exr
- render.011.exr
I find it also easier to read and spot the number you are looking for when there is vertical consistency.
And software vendors often include options to handle this case. Using %04i or #### in the name of the file for example, depending on the situation.
The Project
That said, let’s see how we organize it at Les Fées Spéciales. It is the way we do it, which may not be the most perfect way for other contexts. But it follow the rules we outlined before and it works for us.
Project folder
You probably always have a root folder where your project is stored. Be consistent here as well. You might have to write it very often in scripts or shells, so be concise. At mac guff, the folder for the Despicable Me 2 movie was not despicableme2, but dm2. Or for Dilili in Paris just dilili.
It’s the level 0 of your project tree achitecture. Even if above you might have a store folder, like the mount point (X:, /productions/,…), we’ll focus on from that point.
Within that folder you might find:
- your Library of assets
- your film or films (in the case of episodes)
- probably a production folder with important planning and stuff from or to the client
Our LIBRARY
First level: the Library itself
We start with the library of assets as a movie. In our case we have one library shared for all the sequences and we simply call it ‘LIB’. This is our first level under the project.
Second level: the type of assets
Inside LIB, on the second level, you find the type of assets you are creating:
- chars (for Characters)
- sets
- props
You could also have other kind of assets, like VFX (pre-defined explosions, waterfalls, and stuff you can re-use in several shots)
Third level: the family of assets
The third level is the family of the asset. No matter what kind of asset, we try to have consistency even in the deepness of the folder hierarchy. So for example characters might have main, secondary, tertiary, where props can have vehicles, house, garden,… and sets can also be separated in families like indoors, outdoors, castle (could be the castle and its independent rooms, etc.) or by sequence, for example.
Fourth level: the asset
In the fourth level, this is the asset name. It’s pretty important to define it well and we try to have it with a unique name constraint all across the production. This is easier to handle in production management software also. We keep it as simple as possible. The name of the character, for example.
Fifth level: the working department
Then we just go for the department as the fifth level: Mod (Modeling), Shad (Shading), Rig (rigging),…
So far we have: /LIB/type/family/asset/department
Sixth level: task files
Then you find your production files. Blender files for example. Following the advice listed above, that task file has a full name composed of: LIB_type_family_asset_department_tags(optional)_version.extension. So for example a high resolution shading file of a main character named Tomoe will be: LIB_chars_main_Tomoe_shad_high_v05.blend
Tags are arbitrary, you can have several ones, separated by ‘-‘ (dash) in our case. But any task file is identified easily just by the name.
So the full path of that asset will be : myProject/LIB/chars/main/Tomoe/shad/LIB_chars_main_Tomoe_shad_high_v05.blend
Our FILM
First level: the movie or episode
For the Film folder we follow the same rules. This time our film is not called FILM but a code name. For example, for our production Dilili, the code name was DIL. It could also be, in the case of a series, the episode identifier, like E01, E02,…
Second level: the sequences
In the second level we have the sequences. We name them S followed by the sequence number: S01, S02,…
Third level: the shots
The third level is the Shot level. Named P (for Plan, shot in french) and the shot number: P001, P002,…
It’s trickier in english with “Shot” to differentiate it from Sequence. But you could write the Sequences Seq01 and the shots S001 with a S instead of our P.
Fourth level: the working department
The fourth level is the department: layout, animation, exports, lighting, lighting-render, compo,… We also often have movies, playblasts (openGL renders in blender) or previews of animation or compositing. So we have specific folders to save those movies, suffixed with -movie: layout-movie, animation-movie, compo-movie,…
At that point we have: /FILM/Sequence/Shot/department
Fifth element: task files
The fifth level is the task file. So for the animation of shot 42 of sequence 13 of the movie DIL, we’ll find: DIL_S13_P042_Animation_Main_v07.blend. As for the assets, we have the possibility to add tags (in the example “Main” would be one).
And the full path of that task file should be : myProject/DIL/S13/P042/Animation/DIL_S13_P042_Animation_Main_v07.blend
Exceptions
Within the film folder we also have other stuff than just the sequences. It can be different from one project to another, but this is what you can find in our working directories:
EDIT
We always save an editing in a EDIT folder as the editing is done BY sequence. In that folder you find versioned editing files (Premiere for example) and the relative versioned exported movies.
We have an EDL (Edit Decision List) folder where we saved EDL or XML from the editing, to export the timing of the edit.
You’ll also find here a folder called current, which has a copy of the last playblast/preview coming from the -movie folders of any shot departments. Every time someone does a playblast, a copy of the output is saved in the current folder. It is the folder used by the editing software, so every time you open your edit you have the last version of all the shots, and an up-to-date movie, ready to be reviewed.
PRODUCTION
In the case of episodic projects, like in our case when we do 20 different shorts for a museum, each episode can have a proper art department, a proper storyboard, etc. This is saved by episode in the PRODUCTION folder.
What is inside that folder is “Out of Pipe”, we do not (yet) track naming conventions inside. Let’s say it’s more… artistic.
How to make the crew use it?
Well, when I started working in the small studio In Efecto in France, I had to setup that naming convention and quickly push the users to adopt it. How? Well, you first need to provide creation and opening tools. Wizards that help the user find their marks easily.
Then you want to avoid having people using different conventions. So you can check the files in your production. Simple script daemons running through all the folders looking for wrongly named files. Report them, investigate why, explain to the users how to do, and maybe update your naming convention if needed.
And then, as I did, every time they use an in-house script or add-on, check if the current file name is correct according to naming convention. If not you could either send a warning or, the hard way, make use of that script or tool impossible. They will come to you asking why, you’ll explain the naming convention, fix the name and relaunch the script. And quickly all your production will be following the same rules.
You do not want two or more naming conventions working together within the same project.
Conclusion
That pretty simple organization allows us to handle most case scenarios of linear audiovisual projects. It’s pretty easy to understand for artists, and it’s easy to handle with scripts. In an upcoming article we will present some of the tools we have been building to handle those questions. In the meantime, you can react or ask for details. Thanks!
Other references
- I mentioned Frank Rousseau’s post: A proposal for your file hierarchy
- Kevin from our team wrote a similar article for small projects back when he was at his collective RGBa: file management for a short movie
We’ll update this part with new links. Write us if you know more examples.
