Staff Blogs

Compressor error: "Unable to submit to queue"

If you're getting this error message when trying to submit a batch to Compressor, follow the steps listed here:

Compressor error: "Unable to connect to background process"

Categories: Apple, Software, Final Cut Studio
0 comments Posted Monday June 7 2010 8:57 AM Permalink

Pro Applications Update 2010-01

Apple released the second Pro Apps update for Final Cut Studio 3 yesterday. It includes Final Cut Pro 7.0.2, Motion 4.0.2, Color 1.5.2, Cinema Tools 4.5.1, Compressor 3.5.2 and Qmaster 3.5.2. There are a lot of fixes there, some for critical issues, and also some performance updates too.

Here's the complete list of changes:

Final Cut Pro 7.0.2
* Drag and drop of files from iTunes works as expected with supported file types.
* Fixes the "Preview Unsupported For This Format" message that would occur occasionally with P2 media recorded to a Firestore FS100 recording device from Panasonic cameras.
* Fixes an issue ingesting from P2 cards with spanned clips which contain more than 80 individual elements.
* Fixes a performance issue with transcoding AVCHD to ProRes on Mac systems using the GeForce 9400M graphics card.
* Fixes an issue with some third-party filters that would cause Final Cut Pro to quit unexpectedly on Mac OS X v10.6 Snow Leopard.
* Markers on multi-cam clips are now preserved when edited into a sequence.
* Log and Transfer function is no longer disabled while using the Share function to export media.
* Media Manager will now copy ProRes media created by AJA Ki Pro.
* Multiple fixes to Change Speed functionality including:
* The term "Duration" in the Change Speed interface is replaced with "Smoothness" which describes the function better.
* Speed keyframes no longer snap to other speed key frames. This improves ease of use when dragging to change speed in the timeline.
* Fixes an issue in which removing a reverse speed change with Ripple Sequence enabled would cause a clip to become half its length.

Motion 4.0.2
* Fixes rendering and performance issues that could occur when using Motion projects in a Final Cut Pro timeline.
* Motion is now more stable when operating in low system memory or low graphic card memory situations.
* The default project setting is now set to PAL or NTSC depending on the operating system localization.
* Addresses an issue that impacted the reliability of the Undo command.
* Corrects the rendering of the Checkerboard Generator to the proper orientation when rendering from a 3D camera.
* Fixes an image usability issue that could be encountered on complex projects when rendered in 32-bit float with Motion Blur turned on.
* Exclusively exporting the alpha channel of a Motion project using the Share feature now returns the proper alpha channel.
* An issue with importing files named with dual-byte characters has been addressed.
* An issue with the importing of Japanese language text has been fixed.
* Fixes aspect ratio issues encountered with imported shapes and circle masks.
* A correction to the selection tool now makes it easier to select text.
* Fixes an issue related to the the rendering of soft shadows.
* Addresses issues that could produce significant artifacts when objects would intersect in 3D space in certain ways.
* Addresses issues that could be encountered with images of large resolution.
* A performance related fix was applied to the caustics generator.
* Adjusting paint stroke control points in 3D space is now more responsive.
* Corrects the rendering of anamorphic projects to the digital cinema desktop output.
* Changes were made to certain filters to match the output they produced in Motion 3.
* To address a user interaction issue, the selected animation curve in the keyframe editor now displays on top of other curves.
* Fixes were made that benefit some filters applied to images that are cropped.
* A performance issue with text flattened to a single plane was addressed.
* Fixes an issue that could have affected some third-party FxPlug plug-ins that perform temporal manipulations like retiming.
* Addresses issues which could appear to soften images, or flattened groups of images, of odd-sized resolutions.
* Manipulation issues that could occur with multiple duplicated and cloned objects or groups within a project have been addressed.

Color 1.5.2
* 3D Color Space Scope now allows zooming.
* Resolves an issue in which media would sometimes become disconnected after a save and restart.
* Fixes an issue with CFX nodes not being imported when choosing File > Import > Color Corrections.
* Addresses several issues with the loading and processing of motion trackers.
* Fixes an issue with the green channel on video scopes displaying an incorrect value.
* RED 2K renders from 4K media are no longer softened.

Cinema Tools 4.5.1
* Trying to save a Cinema Tools database to a read-only volume no longer closes the database.

Compressor 3.5.2
* Corrects disk size limitation messages when burning a project to an AVCHD dual-layer disc.
* Fixes issues with long chapter names not appearing correctly in disc templates.
* Fixes issues when inserting a DVD-R or BD-RE disc with data already present.
* Provides warnings before a DVD or Blu-ray Disc is burned when the source will not fit on the media.
* Provides an option to cancel a disc burn when quitting Compressor.
* Fixes issues with burning a DVD using DTS audio.
* Fixes an issue with automatic bit rate so that the appropriate bit rate for a single or dual layer DVD-R is applied.
* Corrects an issue with the encoding of HD video when burning a Blu-ray disc with surround sound.
* Addresses issues with saving and changing audio channels and video using surround sound.
* Corrects a problem that caused Share to not work when switching to another user on the same machine.
* Fixes a MobileMe sharing issue when entering a single character in the "Viewable by Name" field.
* Fixes MobileMe upload for file names that contain the "+", "?", or "%" characters.
* Corrects an issue with uploading an IFF image sequence with an uppercase extension.
* Corrects the default bit rate for the iPod encoding setting from the Share menu.
* Corrects color shift when transcoding image sequences.
* Correct timecode is now displayed in the Preview window when sending a Final Cut Pro sequence to Compressor and applying a timecode generator.
* Job Action "Open Application" now works with multiple targets.
* Fixes an issue with creating a new job with Target Output from an image sequence to a cluster.
* Fixes an issue with source field order for Compressor plugins.
* Fixes a pixel aspect ratio issue with transcoding a duplicated image sequence to a custom TIFF setting.
* Corrects an issue that stopped a WAV file being associated with with an image sequence for output.
* Compressor preferences are now honored when submitting jobs from Final Cut Server.
* Corrects a field order issue with certain sources.
* Fixes an issue with the audio tab becoming inactive when moving between the video and audio tab on MPEG-2 targets.
* Addresses an issue with upload to MobileMe with files containing reserved characters.
* Improves file size estimation for H264 files produced for Blu-ray.
* Corrects an issue when burning a Blu-ray or DVD disc from a Final Cut Pro sequence with 5.1 Surround Sound.
* Fixes an issue with display of password-protected MobileMe movies on iPhone.
* Addresses a pixel aspect ratio compatibility issue.
* Improves stability when processing large batches.

Qmaster 3.5.2
* Fixes a memory leak associated with transcoding to a YouTube destination.
* Fixes unexpected quitting when copying large numbers of image sequences to a destination.
* Corrects issues when copying AVI and reference movie media to a Wide Area Network cluster.
* Corrects an issue with Apple Qmaster command line preferences utility when a user is only logged in remotely.

Be aware, however, that people are already reporting issues with the new update.

Apple also posted the following known issues:
* Final Cut Studio (2009), Final Cut Pro 7.0.1, & 7.0.2 with Mac OS X v10.6: Final Cut may stop responding when capturing DVCPRO HD via FireWire
* Final Cut Studio (2009), Color 1.5: Upgrading a Color 1.0.x project to 1.5.x changes any existing Trackers
* Final Cut Studio (2009), Color 1.5.2: Video Output of REDCODE material clips at 97 IRE
* Final Cut Studio (2009), Color 1.5.2: Unrendered freeze frames in Color displays an unexpected frame

Categories: Apple, Software, Final Cut Studio
0 comments Posted Thursday March 25 2010 2:15 AM Permalink

Combining odd and even frames in Shake

I've just finished helping a guy with a Shake issue via email and I think the solution would be useful to others. He had a situation where he had one movie clip with all the odd frames from a sequence and another clip with all the even frames, and he wanted to combine them into a single file.

At first glance, it would appear to be as simple as using the Select node to switch between them, however you would miss out half of the frames by doing this. Here's the solution that worked for him:

1. Import the two files into Shake via the FileIn node.

2. Add a TimeX node to the first clip and set it to 0.5*time (this doubles every frame) and do the same for the second clip.

3. Add a Select node and plug the output of the two TimeX nodes into it. So that it's not confusing later on, make sure the output of clip 1 goes to the first input and clip 2 goes to the second.

4. In the Branch parameter of the Select node, set a keyframe at frame 1 with a value of 1 (i.e. the first input), then step to frame 2 and keyframe the value to 2 (the second input).

5. Load the parameter into the Curve Editor by clicking the clock icon next to the Branch field so that a tick icon appears, and then click the Curve Editor tab.

6. In the Curve Editor, change the Cycle parameter to MirrorValue.

7. When you render the FileOut, make sure to double the frame range as the sequence is now twice as long.

Categories: Compositing, Software, Visual Effects
2 comments Posted Saturday November 7 2009 6:27 AM Permalink

Speeding up Compressor

Is Compressor taking too long to encode? Here are some tips to speed it up.

Don't export directly to Compressor from Final Cut Pro

Although sending your timeline directly to Compressor may seem faster because it cuts out the step of exporting as a QuickTime movie, it is much slower overall because Compressor needs to request each frame one-by-one from Final Cut Pro instead of just reading the data out of the movie clip. It gets even slower if you're using Frame Controls or doing multi-pass encoding.

Just go to File > Export > QuickTime Movie (not QuickTime Conversion) and make sure Setting is set to Current Settings and Recompress All Frames is switched off. This ensures that the movie will not be recompressed unnecessarily.


Here's a tip - if most of your timeline is already rendered, deselect Make Movie Self-Contained. This creates a small reference movie that links to the render files on disk instead of writing the data into the file. This will be much quicker to export. If your timeline is not rendered, however, this setting is unlikely to offer any speed advantages over a self-contained movie.

Once the file is exported, drag it into Compressor and set up your batch as normal.

Only use Frame Controls where necessary

Frame Controls allow you to improve the quality of resizing, retiming and deinterlacing operations, however they should only be switched on when you are actually performing one of these operations.


In addition, you should use Better quality instead of Best, as Best is usually reserved for extreme circumstances and in most situations gives you a massive performance hit with no benefit.

Perform heaving-lifting in a separate pass to the encoding

Using Frame Controls with a multi-pass or long-GOP encoder can really slow things down because the Frame Controls processing will need to be reapplied to every frame each time it is read. Even if you're not using Frame Controls, compressing from a processor-intensive codec such as HDV or H.264 can seriously slow things down.

It makes sense, therefore, to perform all these processor-intensive operations on an intermediate movie clip (ProRes would be a suitable codec for this) and then drag in the intermediate clip and encode it to the desired format.

But there is an easier way. Job chaining is a little-used yet very powerful feature of Compressor that allows you to plug the output of one job into the input of another for additional processing.

Here's how to do it:
1. Drag your movie clip into Compressor.
2. Drag either the ProRes 422 for Interlaced Material or ProRes 422 for Progressive Material (depending on your source media) setting onto the job in the batch window.
3. Go to Job > New Job with Target Output. This will add a new job to the batch with a chain symbol to show it is linked to the job above it.
4. Drag your desired output setting onto the chained job and set up the destination as normal.
5. Submit the batch

Use a cluster

A cluster is a group of computers where each machine processes a portion of the movie simultaneously, potentially providing a massive speed boost. This service is provided by Qmaster and can be set up in the Qmaster section of System Preferences or the Apple Qadministrator utility.

You will need Qmaster set up on each computer in the cluster, and all computers should have the required codecs installed and have full access to the source media and cluster storage area.

If you have a multi-core computer, Qmaster can emulate the effects of a cluster by launching multiple copies of Compressor side-by-side to process the movie in a "virtual cluster". This can make a dramatic difference to encoding times. I wrote a tutorial on this a while back.

Conclusion

Compressor is not the fastest encoder in the world but with these tricks you can make it a lot more bearable. One important trend to note is that the simplest method isn't always the fastest - with a little extra setup beforehand you can save a significant amount of time overall.

Categories: Video Editing, Software, Final Cut Studio
3 comments Posted Saturday October 17 2009 3:47 AM Permalink

How to run Final Cut Studio 3 on a MacBook

I've been meaning to post this for a while but haven't had the chance until now. I had to be an early adopter of Final Cut Studio 3 because I develop software that interfaces with it, but I had no desire to edit on a x.0 release so I decided to install it on my MacBook solely for testing.

The only problem is that, unlike FCS 2, the FCS 3 installer refuses to continue because it requires a 128 MB graphics card. Only Motion and Color are heavy on the graphics card, so the other applications in the suite will work just fine on a lesser machine. Luckily the block is easy to work around.

Here's how I got it working on my MacBook:

1. You will need to change one of the files on the DVD. As you cannot write directly to the DVD itself, you will need to create a writable copy of the disc. Put in the Final Cut Studio DVD and open up Disk Utility (located in /Applications/Utilities).

2. Select the disc in the left-hand pane and click New Image in the toolbar.



3. Set Image Format to Read/write and Encryption to None.



4. Type a relevant filename and click Save. It will take several minutes to process.

5. Open up the disk image and ctrl-click on Install Final Cut Studio. Select Show Original.



6. A new window will appear with the FinalCutStudio.mpkg file selected. Ctrl-click on this file and select Show Package Contents.



7. Navigate to Contents/Resources and ctrl-click on the Requirements Checker bundle. Select Show Package Contents.



8. Navigate to Contents/Resources and open up minsys.plist in Property List Editor (if you have the Apple developer tools installed) or TextEdit.



9. Under the heading AELMinimumVRAM change the number from 128 to a number lower than or equal to your current video memory. I changed mine to 32. Alternatively you could change block to warn and the installer will warn you but let you continue installation.



10. Save the file, close all the folders that appeared and double-click Install Final Cut Studio on the disk image. You should now be able to install Final Cut Studio without problems.

Some of the applications in the studio also contain a minsys.plist file that will need to be modified. These applications are Final Cut Pro, Motion, Color, Soundtrack Pro and DVD Studio Pro (even though the latter apparently has no changes from FCS 2).

If you have FCS Maintenance Pack installed, you can do this quickly and easily by checking the Low Minimum Requirements option in System Toolkit. Otherwise follow the steps below.

11. Ctrl-click on the application in question and select Show Package Contents.



12. Navigate to Contents/Resources and open minsys.plist in either Property List Editor (if you have the developer tools installed) or TextEdit.



13. Under the heading AELMinimumVRAM change the number from 128 to a number lower than or equal to your current video memory. I changed mine to 32. Alternatively you could change block to warn.



14. Color will still warn you about your screen resolution - however you can just select Never show again and continue.



That's it! You will need to repeat steps 11-14 whenever you update Final Cut Studio.

Note: Some people have suggested simply removing the Requirements Checker application but I do not advocate deleting files.

Categories: Apple, Software, Final Cut Studio
1 comment Posted Wednesday August 12 2009 4:30 PM Permalink

Mac OS X 10.5.7 released

Apple has finally released the much-delayed 10.5.7 update to Leopard. There was much speculation that it was delayed to coincide with the full release of Safari 4 which is currently in beta, however this is clearly not the case as Apple has just released Safari 3.2.3 and an updated beta of Safari 4 (you won't see this in Software Update until you update to 10.5.7 though).

The release notes are quite brief, however it is possible to get a much better overview by looking at World of Apple's seed notes for the 10.5.7 beta. This is not a comprehensive list but gives a good indication of the final changes.

There's some good stuff in there - lots of networking fixes and several changes aimed at improving sleep/wake reliability (I've actually stopped putting my MacBook to sleep because it either wakes up constantly every 10 mins or never wakes up at all). It also contains updated graphics drivers. There's really nothing serious in there so you can update at your leisure. It's very much a polish update.

If you are thinking of updating though, remember the Golden Rules:
1. Never update in the middle of a project
2. Wait a week or two for problems to surface
3. Perform a clone of your system before installing
4. You will experience greater reliability from the larger combo updater

10.5.7 Delta Update (10.5.6 -> 10.5.7)
10.5.7 Combo Update (10.5.x -> 10.5.7)
10.5.7 Server Delta Update (10.5.6 -> 10.5.7)
10.5.7 Server Combo Update (10.5.x -> 10.5.7)

The bundled security update is also available separately for Tiger users as Security Update 2009-02.

Update: Mac Mini users are reporting problems already. It seems like certain DVI displays are not being recognized correctly on Mac Minis.

Categories: Apple, Software
0 comments Posted Tuesday May 12 2009 4:26 PM Permalink

Best and worst of iPhone 3.0

Apple released details of iPhone OS 3.0 yesterday. It's got all kinds of great features - turn-by-turn GPS, copy and paste, MMS, tethering, to name a few. Here's my take on the best and worst features.

The Best

Stu Maschwitz from ProLost was particularly impressed with the new accessory APIs, which is probably the biggest new feature that no-one is talking about. Do not underestimate the importance of this - it is HUGE. This bridges the gap between software and hardware and takes the phone from a simple smartphone to a full-scale development platform.

No other phone developer is offering this. A lot of them are jumping on the app store bandwagon now but Apple just increased the gap even further. Apple is once again taking advantage of their control over hardware and software - I believe Apple's toughest rival is Google but it would be difficult for Google to replicate this with Android as they do not control the hardware.

Of course, hardware development is more complicated from a business perspective as you'd need to manufacture it, handle billing, arrange shipping and keep inventory (which is always a little risky, particularly in a recession). So this is purely aimed at companies and not bedroom developers. I don't know if Stu will get his wish for an iPhone scope (I hope so) but it will be very interesting to see what developers come up with. There are loads of on-set applications here ranging from remote operation/focus pulling to synchronized iPhone timecode slates to an app for editing camera/lens metadata. I could totally see the RED guys embracing this technology.

The Worst

The most worrying aspect is the new in-app payment system. I can picture only a few scenarios where this would be genuinely useful (such as an e-book reader or GPS app with its own maps) and I can see a lot of developers abusing this. It really will bring out the worst in developers.

The problem with the iPhone development platform is that it's far too easy for developers to charge money. If they were selling software on their own sites they would need to pay for marketing and sort out a system for billing. It's a lot more complicated and there are more up-front costs so they'd need to ensure they were selling something of value.

With the iPhone it is very easy for someone to create a poor application in five minutes and charge for it, and if some poor sucker buys it that's a win for the developer. If no-one buys it, then the developer hasn't lost anything either. It's win-win for them.

There are lots of applications on the app store for $0.99 that are completely valueless (for example the multitude of "flashlight" apps that display a white square on your screen). The devs are charging $0.99 because they can. It's too easy. And Apple's just made it even easier. I see this as a bad thing for the consumer.

Although I am certain this will be abused, I think the benefits of the accessory APIs will far outweight abuse of this feature. If developers need to keep inventory, they will have to ensure they have something worth selling. Sure, not every developer will choose to use in-app purchasing, and I'm sure many will offer excellent value for money. But when the new apps are accepted into the store, it will be important to read the small-print and be aware of exactly what you are getting for your money.

Having said that, I can't wait to see what developers come up with.

Categories: Apple, Software, Analysis
2 comments Posted Wednesday March 18 2009 4:34 AM Permalink

Guide to Final Cut Pro Internal Tools

Did you know Final Cut Pro has a hidden menu? Just hold down Cmd + Option + Shift and click the Tools menu. You should now see a new menu item called Internal Tools.



This is a debug menu used by the developers of FCP, but it actually has some uses for us mere mortals as well. It's especially useful for plugin developers and system troubleshooters, but some features benefit all users. Here is a rundown of what's in the menu.

Adjust Windows

When you open the Audio Mixer, Frame Viewer and several other tools in the Tools menu, they open in a tabbed window called the Toolbench. This normally appears on top of the Viewer and is hidden behind it when you select the Viewer again.

The Adjust Windows menu item adjusts the window layout so the Toolbench can fit in without being obscured by other windows. If you don't have the Toolbench open when you click this item, it will leave a gap in your window layout.

Before


After


To restore the default window configuration again, press Ctrl+U.

App / Perf Info

This generates a text file containing just about everything you could ever wish to know about your FCP configuration.



Probably the most useful part is the Project / Sequence Information section. This gives various statistics about your project including the total number of clips placed in your timeline and the proportion of those clips that were unique. It will even tell you how many edits you performed per minute (don't let your boss see this!).

Log OpenGL Profile Costs

This function displays timing information about OpenGL Profiling in the Console. It will do nothing until you start profiling (see the Statistics section below), at which point it tells you how long (in ms) it took your machine to process the given frame sizes and codecs.



One thing I noticed about this is that it takes a fair while for the data to be posted to the Console - around 1-2 minutes.

Log VM Footprint

This outputs the current memory usage to the Console.



Resident refers to data in physical memory, Virtual refers to data on the disk ready to be copied into main memory as required. Note that the system often assigns more virtual memory space than it needs, so not all of that space will actually be used. So if you see a ridiculous number like 4 GB, it is not necessarily a cause for alarm.

Movie Dumpster

This one crashes Final Cut Pro for me, so I wasn't able to ascertain exactly what it does. However, Apple offers a tool called Dumpster on their developer site which gives you a large amount of technical information about a specific movie clip. I would imagine this would be similar if it worked.

Playback Meter

This will show you how your media drives are performing, and is probably the most useful tool in the menu.



If you are not using a RAID, only the audio bars will work. Everything is fine while the bar is green, but if it turns yellow or red, this is an indication that your computer cannot keep up.

Screen Size Override

This allows you to fool Final Cut Pro into thinking that you have a different-sized monitor.

Just select the desired resolution from the pull down menus and press Ctrl + U to automatically resize the windows to the new dimensions.



If you mess things up, just restart Final Cut Pro and press Ctrl + U to restore the windows to the correct position for your screen resolution.

Statistics

This displays a lot of technical information about what's going on under the hood of FCP.



For instance, select the Render check box, drag the playhead across the timeline and then click the Average button. This will average the times taken to perform the various function calls required to render the image to the screen. If your system is getting sluggish, you may find it useful to use the Averages function to see which areas are taking a long time to process.

Most of the checkboxes are self-explanatory so I won't elaborate on them here. However, I would advise selecting them one at a time otherwise you'll end up with an overflow of information.

If you want to clear the text box, click Reset. Copy copies the entire data to the clipboard and Copy Avg copies only the averages. Unset All deselects all of the checkboxes.

Both Auto Profile and RT Verify go through each item in the Browser (video clips, sequences, images, etc), playing each one four times and averaging the data. Sequences take longer because the profiler initially tests with only the V1 track enabled, and then gradually enables V2 and V3 to perform compositing tests (if such tracks exist of course).

It's worth mentioning that since it goes through everything four times, it will take a while to finish processing, depending on the size of the project. You cannot switch to another application either or the profiling will fail. I would advise creating a brand new project with short media clips and sequences. If you truly have serious bottlenecking, this will still be apparent in the shorter sequence.

All the timing information is stored inside the Profile.csv log. Although you can view this in TextEdit, it will be difficult to read because it will not be formatted correctly. A spreadsheet application, such as MS Excel or the free NeoOffice or OpenOffice packages, is recommended.



If you are experiencing poor performance, look at the timings in this file to see where the major bottlenecks are. For instance, a slow FXScript filter could be indicative of a CPU bottleneck or an inefficient script. A slow Flop operation could be indicative of a GPU bottleneck.

Auto Profile and RT Verify appear, on the surface, to perform exactly the same task. However, it is likely that Auto Profiling actually updates your RT information whereas RT Verify merely performs the test without changing anything. This is difficult to verify without official documentation though.



Most of the functions in the Internal Tools menu will be too technical for a lot of users, and so the menu will likely remain something of a novelty. But if you are troubleshooting performance issues or developing applications and plugins for FCP, it is an invaluable source of information on the inner workings of Final Cut Pro.

The only downside is that the menu must be reactivated every time you start Final Cut Pro. If anyone knows of a way to enable it permanently through a preference file, I'd love to hear about it.

Categories: Software, Final Cut Studio, Analysis
8 comments Posted Wednesday March 11 2009 4:48 PM Permalink

Compressor error: "Unable to connect to background process"

The "Unable to connect to background process" and "Unable to submit to queue" messages are common errors that unfortunately have many causes.

Before you do anything else, check that your version of Compressor is compatible with your operating system. Only the versions included with FCP 5.1.x and 6.0.x are compatible with OS X 10.5. If you have an incompatible earlier version, you will need to either upgrade Final Cut Studio or downgrade your operating system.

Try the following steps:

1. In Compressor, go to Compressor > Reset Background Processing.



2. Run Compressor Repair.

3. Trash Compressor preferences (can also be done in Compressor Repair).

4. Try these steps (use at your own risk).


Do the following if nothing else works:

5. Disable your internet / network connections.

6. Reinstall Compressor and Qmaster.

7. Perform a full erase and install of the operating system and reinstall from scratch.

Categories: Apple, Software, Final Cut Studio
3 comments Posted Saturday February 14 2009 12:03 PM Permalink

Getting to Know the Terminal Part 2: More File Operations

In the previous article, I showed you how to launch the Terminal and perform basic tasks like navigating through folders and dealing with files. Today I will build on this with more advanced file operations, working with directories, and wildcards.

Working with Directories

To create a new directory, navigate to the relevant parent directory using the cd command and type the following:

mkdir MyNewFolder

Remember: as discussed last time, if you want a file name with spaces, you must do either of the following:

mkdir "My New Folder"

or:

mkdir My\ New\ Folder

What if you need to create several new directories? Simple:

mkdir Dir1 Dir2 Dir3

A little-known feature of the mkdir command is that you can create multiple directories with just one command. Just separate them with spaces.

If you want to create a long chain of directories (e.g. Renders/Video/TIFF) you can use the -p parameter like so:

mkdir -p Renders/Video/TIFF

Press Enter and it's all done, instantly. Imagine how long it would have taken to do that within the Finder GUI! This is why the command-line is still used in the 21st Century even though we have pretty GUIs to look at.

To remove a directory, use the rmdir command:

rmdir Renders/Video/TIFF

The above command assumes that all directories are empty. If they are not, an error will be returned. To remove a directory and all its subfiles and folders, use the following command:

rm -dR Renders

(the d parameter tells it to include directories, R tells it to be recursive)

Wildcards

Last time I mentioned file operations such as cp for copying and mv for moving files. But what if you need to perform an operation on a large number of files at once?

Wildcards can be used to substitute characters. So if you want to copy, say, 100 files that are named Image.001.jpg to Image.100.jpg, you would use wildcards to substitute for the numbers. This will allow you to copy all of the files with one command instead of a hundred like so:

cp Image.*.jpg Renders

Question mark (?) substitutes a single character. So for instance, if you had files called render_v1.iff, render_v2.iff and render_v10.iff and you typed ls render_v?.iff, it would return only renders 1 and 2 because they are a single digit whereas render 10 is two digits.

Asterisk (*) substitutes a potentially infinite length of characters (including no characters at all). Use this if you don't know the precise length of the substitution. In the example above, if you type ls render_v*.iff, it will return all three files because it matches both the 1- and 2-digit numbers. You can also type, for example, *. jpg to return all JPEG images or *.* to return all files in the directory.

So, going back to the 100-frame image sequence previously mentioned, you would type ls Image.*.jpg to return a list of all of the images. That's all well and good, but what if you don't want to return all of the images - what if you only want a specific range?

Braces ([ ]) can be used to be more specific. In the above example, type ls Image.[20-30].jpg to restrict the results to frames 20-30. You can also restrict characters in the same way, such as [b-f] (remember that everything is case-sensitive).

If your numbers or letters don't fall into a sequential range, you can pick a set of non-sequential numbers or letters such as [brz] or [179] to match any of these characters. You can also combine them with ranges like so: ls Image.[1-10,12,14,20-40].jpg. This will return frames 1-10, skip frame 11, return 12, skip 13, return 14 and display frames 20-40.

And if you don't want specific characters to be returned, use the following syntax: [^bgv].

Combining the examples above: if you want to match a file starting with the letters a-c or x-z, with three miscellaneous characters in the middle, ending with a three-digit number at the end, you would type ls [a-cx-z]???[0-9][0-9][0-9].*. As you can see, it gets complicated pretty quickly.

But imagine if you had to search through and move or copy those files manually. It would be time-consuming, tedious and prone to human error. The command line comes into its own when you want to perform operations on a large number of files at once.

Editors can get away with not knowing this (although it can be very useful for them to know) but it is required knowledge for anyone who wants to get into visual effects.

I had originally planned to cover permissions here but I'm going to move it to the next article because I went into a lot of detail and it's too large to add on the end of this one. I'd much rather delay it than miss out information.

So next time I will be covering file permissions, symbolic links and opening, viewing and saving text files.

Categories: Apple, Software, Visual Effects
0 comments Posted Monday January 19 2009 10:26 AM Permalink