Cut Schema v1

Historically, Shotgun has tracked Cuts in a simple way, focusing primarily on the latest information.  While we make the most of what we do track in apps like Screening Room, there is certainly more that can be done.  With a growing emphasis on review features in Shotgun, it's time to start thinking about the larger feature set for collaboration with editorial.

We see bottlenecks around editorial with many of our clients and feel this is an area where we can help bring about big improvements to production workflows.  Ultimately, we want to provide the ability to take important cut data from any tool that has a timeline, extract the key information, and bring it into Shotgun so users can leverage it to its fullest.  Once in Shotgun, we can then supply this information to other apps, with Shotgun acting as the interop medium bridging the Cuts gap.

Having this information in Shotgun opens up many more feature possibilities: tracking Cut histories; managing multiple Cuts, each of which might be current, like with the :30/:60/:90 case in Commercials; watching any piece of media in the context of the Cut, including non-VFX Shots; tweaking a Cut and sending it back to editorial.  We believe these should all be part of Shotgun.

That said, the full feature set and priorities are a larger conversation.  As a starting point, we want to kick things off by discussing the schema and outlining how Cuts should be structured within Shotgun.  Getting the schema right is key, and it's important to make sure our clients' Cut tracking needs and point of views will be covered, since it will be the basis for the feature additions.  This post will go through our current thinking on a proposed Cut schema for the sake of community discussion.  Please let us know your thoughts in the comments and reach out if you want to have a direct chat. 

 

Approaches

A few of us on the Shotgun team have experience with developing Cut pipelines at previous jobs, and we also met with a number of clients to both learn about the setups they built themselves and talk through workflow details.  Some patterns emerged, and we observed two clear patterns in approaches taken for integrating Cuts with Shotgun:

  1. Separate Entities for Cuts
    Shotgun has legacy Cut and CutItem entities that were never fully fleshed out.  Even so, some clients have run with them and shaped them into something they can use on their own.
  2. Tracking Cuts as Playlists
    Other clients oriented things around watching Cuts via Playlists, adding on the additional data they needed to support editorial features.

We went back and forth internally on which direction was the right one, and in the end, we realized that both are viable and should be presented to the community to get the group's thoughts.

 

Separate Entities

Because there are clients who might still be using the legacy Cut and CutItem entities, we thought it would be easier to view this approach as using new, additional entities.  In terms of naming, we gravitated to the idea of calling the new entities EDL and Edit, where we introduce the concept of a “Shotgun EDL” that EDLs in other shapes and forms (CMX, FCP XML, etc.) can be converted and mapped into.  That said, we aren't married to these names and are all ears to better ideas!  For the purposes of this discussion though, we will refer to them as EDL and Edit. 

EDLs:

The EDL entity would represent a set of editorial information, where each record tracks an iteration for Cut history purposes.  The EDL would also contains fields that are global to the edit information being brought in to Shotgun.  Here is a rundown of the proposed set of default fields with types in parenthesis (additional custom fields could of course be added on top of them):

blog_edl_entity.png

  1. Name (text).
  2. Thumbnail (file).
  3. Status (status).
  4. Description (text).
  5. Type (list).
    • This field would denote things like Client vs. Internal, :30 vs. :60 vs. 90, etc.
  6. Link (entity or polymorphic multi-entity).
    • This field would usually link to a Sequence, but it could link to other entities based on client workflows like Scene, Episode, etc.
    • It could also be a “Links” field as on Notes, where all linkages would be reflected in the field (ex. the Sequence that the EDL represents as well as each of the individual Shots within the Sequence/EDL).
    • blog_links_field.png
  7. Timecode Start (timecode or text).
  8. Timecode End (timecode or text).
    • These fields would preserve relevant data from the incoming EDL.
  9. Duration (duration).
    • Clients seem to have different display unit needs, so while the initial thought was to display in seconds across the board, more robust field options that allow clients to specify the display units (seconds vs. frames vs. …) on a per-page basis would be a nice feature add here.
    • This notion applies to any duration field in this document.
  10. FPS (float).
    • This field would apply whenever playback referenced this particular EDL.
  11. User Restrictions (multi-entity w/HumanUser).
    • Private cuts came up a lot in our client meetings, and this field would drive that feature.
    • The behavior would follow the way Pipeline Configurations work in Toolkit, where a restriction can be set to specific People when linked, but a blank value means anyone can see it.
  12. Uploaded EDL (file/link).
    • Having access to the original EDL that a tool processed can be very useful for a variety of scenarios, so this field allows it to be stored in Shotgun.

Edits:

Next we have the Edit entity, which represents each item in the EDL.  The Edit entity stores all of the information required to reconstruct the timeline for playback, including which clips were present at the time of EDL publish.  As part of this, the concept of a "base layer" will be introduced in Shotgun, which is a Version that represents the entire EDL, and used as a placeholder for non-VFX ShotsA in context as well as a fallback for items that may not have a valid Version yet in the early stages of a ProjectB.  Edits can also link to the same entity and Version for the case where something appears twice in the same EDL (a.k.a double cuts)C:

blog_edit_entity.png

  1. Cut Order (number).
    • A sort field based on the EDL.
  2. Link (entity).
    • This field would usually link to a Shot, but it could link to other entities based on client workflows like Scene, some CustomEntity, etc.
  3. Version (entity).
    • This field stores the Version present at the time of EDL publish for both historical tracking and so playback can potentially use it when building a Cut for review (if desired).
  4. Timecode Cut In (timecode).
  5. Timecode Cut Out (timecode).
  6. Timecode EDL In (timecode).
  7. Timecode EDL Out (timecode).
    • These fields would preserve relevant data from the incoming EDL even though tools will likely parse them and translate them to frame numbers for playback.
  8. Cut In (number).
  9. Cut Out (number).
    • These fields represent the actual frame counts that playback should use.
    • They can start at any number and should reflect the proper frame ranges being used by artists when generating new media.
  10. EDL Cut In (number).
  11. EDL Cut Out (number).
    • These fields represent the overall frame counts for the EDL in relation to each Edit.
  12. Edit Name (text).
    • A name field comes standard on all entities but doesn't necessarily add much for this use case.
    • We suggest automatically filling it with something sensible, like the name of the EDL appended with the cut order value.
  13. EDL (entity).
    • This field links each Edit back to the EDL (not pictured).

There are more fields on the Edit entity to go through, but one question we came up on while putting this proposal together was whether or not all data from the original EDL should have a place in Shotgun.  As a means of keeping the schema as light as possible, we felt it would be worth asking if fields that aren't likely to be used for playback, like these timecode fields, should be official parts of the schema.  The initial feedback is that a 1:1 mapping is a good thing, but please let us know your thoughts one way or the other.

Audio:

While Screening Room doesn't support audio yet, it's a feature we expect to add as part of the overall feature set (priority TBD!).  Our proposed schema accounts for audio as a foundation for audio support:

blog_audio.png

  1. Path to Audio File (text).
    • We propose tracking the location of audio files in a text field, the same way we currently track Path to Movie and Path to Frames.
  2. Audio Offset (duration).
    • This field denotes the offset value between the start frame of the audio file and the start frame of the Edit since that may not always be the same.
    • A blank value will be treated as zero.

There are a few additional things worth mentioning with regard to audio:

  • Audio files may have a hierarchy, and might be defined at different levels, i.e.
    • Per Edit.
    • Per EDL.
    • Per Version.
    • Per Version Link (ex. Shot).
  • Cuts playback follows this hierarchy.
    • Or, at least some subset of this hierarchy.
    • Is this overkill?
  • Non-Cuts playback just uses Version and Version Link.
    • In this case, there wouldn't necessarily be an EDL or Edit to get the audio from.
  • Start with “Path to …” text fields as in Screening Room.
    • As Screening Room evolves to support multiple representations and Published Files, audio support could transition along with it to the same structure.

FPS and Retime:

Similar to audio, there are fields to track FPS and retimes for future feature support:

blog_fps_retime.png

  1. FPS (float).
    • This field would apply whenever playback referenced this particular Edit.
  2. Retime (text).
    • This field would contain text that represents a curve to use for the retime and is mainly here to gauge client reaction.
    • Admittedly, this is a bit “handwavy” in our proposal, but a curve syntax would be decided upon for actual feature release.
    • We wouldn't expect this field to be shown much in the UI (aside from debugging), and it would exist to supply tools with the information needed to playback a retime.
    • The retime could also potentially represent an FPS change, negating the need for that field, although our sense is that having a dedicated FPS field is clear and nice.

Aside from the curve syntax, there a few additional things to note on FPS and retime:

  • If an Edit has a retime defined, it will always be used for playback.
  • FPS may have a hierarchy, and might be defined at different levels, i.e.
    • Per Edit.
    • Per EDL.
    • Per Project and/or globally in Shotgun.
    • Per viewer application (ex. RV).
  • Cuts playback follows this hierarchy.
    • Or, at least some subset of this hierarchy.
    • Is this overkill?
  • Non-Cuts playback just uses Project/Shotgun and viewer application.
    • In this case, there wouldn't necessarily be an EDL or Edit to get the FPS from.

Transitions:

Transitions are another thing clients ask about frequently, and Edits will have fields to prepare us for the features to come.  While some EDL formats like FCP XML treat transitions as an item, our sense is that clients wouldn't react well to our schema matching that, based on past experience with the idea of adding extra rows to a list/grid page like our example.  Tracking transitions as fields also lets us simplify the schema a bit to target our likely use cases, making it easier to understand and follow along with what's happening when looking at Shotgun:

blog_transition.png

  1. Transition To (entity).
    • This field links to the Edit that the current Edit transitions into.
    • It will almost always be the next Edit in terms of cut order, but the field makes that explicitly clear and also allows for queries based on this criteria.
  2. Transition Type (list).
    • The list options would reflect supported transitions (ex. wipe, cross-dissolve).
  3. Transition Start (number).
    • The start frame for the transition.
  4. Transition Duration (duration).
    • The total duration of the transition.

Metadata:

Last but not least, there are more complex EDLs out there with additional and potentially arbitrary metadata.  To cover this case, we propose a catch-all field to store these values:

blog_metadata.png

  1. Metadata (text).
    • Your metadata goes here!

In this example, there is a color transform per Edit, and unlike the timecode use case (hypothetically, see above), we want to preserve this data from the EDL for a reason since we might want the color transform to be applied per Edit as part of playback.  The metadata would be the only built-in field to cover this area, but clients can create custom fields to separate the metadata out if desired.

Pros and Cons:

  • Pros:
    • Editorial information is cleanly separated from review information, helping to avoid confusion over where to find it.
    • The schema for playlists can remain simpler.
    • Managing permissions would be more straightforward if centered around entities.
  • Cons:
    • Most playback tools already use Playlists.
      • Tools would need to interact with both entities…
      • EDL and Edit data would need to be pushed into Playlists when needed for playback, meaning Playlists might need the editorial fields after all.

 

Playlists Only

Now we'll move to the Playlist approach, where we won't necessarily need to go into the same level of detail since in a nutshell this entails the same fields just with different placement on the entities.

Playlists:

Here we have our same example EDL, this time as a Playlist.  In this scenario, Playlists get all of the same fields above we proposed for EDL, with two notable distinctions:

blog_playlist_edl.png

  1. Type (list).
    • With the Playlist approach, the idea of Type moves up a level.
    • This field describes what kind of Playlist we have, which might be Revew vs. EDL, to differentiate traditional Playlists made by hand vs. one stemming from editorial.
  2. Reel (entity or list).
    • Since we reserve Type to organize the different kinds of Playlists, this field serves the purpose of the EDL type in the other approach.
    • This could be an entity field or a list field.
      • An entity field might be particularly nice for the Commercials :30/:60::90 case, where a client might want all of the Shots in the same Sequence, but may also want a way to know which Shots are in the :30, which Shots are in the :60 (which could have Shots not in the :30), and so forth.
      • We could also use an entity field in the separate entities approach above as well, if desired.
    • Reel may not be right word for it, but it made sense to tell the story, and we are open to better name suggestions!

PlaylistVersionConnection:

The Playlist approach orients around one entity rather than two, so the proposed field information stored on the Edit entity migrates over to the connection between Playlists and Versions.  However, Playlists have a shortcoming that will need to be addressed if this approach wins out - Version currently can't be in the same Playlist more than once, which affects support for both the base layerA and double cutsB concepts.

blog_playlist_version_edit.png

Pros and Cons:

  • Pros:
    • Easier for (existing) playback tools to work with editorial information.
    • No need to deal with additional entities.
  • Cons:
    • Playlist schema becomes more complex.
      • Separating EDL vs. Review Playlists could be confusing to users.
    • Need to ensure additional editorial schema wouldn't cause any conflicts or client issues.
      • Unlike Cut and CutItem or new entities, almost all clients use Playlists already.

 

Odds and Ends

With either approach, there are a couple last bits of functionality to cover, which come up frequently in client conversations:

  • Cut fields on Shots will remain in place.
    • Clients do not have to use the new schema to define values at the Shot level directly.
    • We will also need to provide mechanisms in Shotgun to help (selectively) update the Shot fields from the new schema.
  • How many times has the Cut changed?
    • This is a common production stats question!
    • Can this criteria be automated?
      • Editorial information can change frequently, and only some changes qualify as “accepted” changes that the stats should reflect on report pages.
      • Does the schema need anything extra to help track this and if so, what should that look like?

 

Timing

We intend this post to act as an RFC (request for comments) on the the Cut schema, so please fire away with any and all feedback in the comments!  We want to continue the discussion with the community through the coming weeks and months, iterating on the spec and nailing it down for the Shotgun 5.3 update in February.  The schema will precede our feature work, so clients can get a head start on their own integration if desired starting with that next full Shotgun release.  Keep tabs on the discussion here in this thread, and depending on how many changes there are, we may do another post with the refined and finalized spec before the 5.3 release.  

 

Follow

21 Comments

  • 0
    Avatar
    Doug MacMillan

    Thanks Ryan!

  • 0
    Avatar
    Dave Lajoie

    Hello guys, we have been reading the proposal and quite happy about it. Is there a schema diagram available for this?

  • 0
    Avatar
    Ryan Mayeda

    Hi Dave!

    Right now, all we have is what's in this doc in terms of available materials.  But, once we lock the schema in, I could also do a diagram for it.

    Glad you're happy with the proposal though.  Are you leaning in one direction or the other in terms of having separate entities vs. the playlist-only approach?

    Ryan.

  • 0
    Avatar
    Doug MacMillan

    Here's a diagram, rough... http://goo.gl/9tp8Yd

     

  • 0
    Avatar
    Doug MacMillan

    Diagram made with txt file attached. Sorry, no UML yet ;-)

  • 0
    Avatar
    Tony Barbieri

    We are leaning towards the Entity approach with a caveat: Could the names be simply Edit and EditItem?  The name EDL is already used to describe a file format so it could get confusing in conversation about what is being referred to...Maybe I'm being overly cautious.  My initial feeling is having two Entities will feel more natural when interacting with them via the API and integrating in other parts of the Pipeline with Toolkit.  It would also avoid bloating Playlists, leaving them to be simple queues for managing review sessions and not trying to fill multiple roles.

  • 0
    Avatar
    Dorian Fevrier

    Thanks a lot! This will really help us! :)

  • 0
    Avatar
    Toke Jepsen

    +1 on Tony's suggestion from here.

  • 0
    Avatar
    Doug MacMillan
  • 1
    Avatar
    Armando Ricalde

    I was reviewing this again with my team, and we prefer the new Entities approach, and I agree with Tony regarding naming them as Edit and Edititem to avoid confusion.

    Right now when exporting and creating Shots from Hiero with our Tank App we are already using a "base layer" but for each Shot by publishing a Version taken from the workprint quicktime file provided by the editor, or the animatic/previs video in case of full CG. Then usually the second Version that is published (from Hiero again) is the color graded plates in case of live action project (and also exported as hi res image seqs to the filesystem of course), or the raw camera material if the grading is taking longer, this way we can even have different color corrected Versions of the material. But we have to create all of the Shots in Shotgun in order for this to work on Screening Room even if there are ones without VFX/CG. If we were just using Hero, the "base layer" is already there, but our artists are using RV and Screening Room for reviewing and they love it, so all this was just to say, thanks for considering a base layer.

    Regarding transitions, in you mockup https://toolkit.shotgunsoftware.com/attachments/token/hes0bomxjgggbuv/?name=blog_transition.png you are not considering the overlapping frames needed to do the transition. As I understand your "Cut In" and "Cut Out" fields are the "Source In" and "Source Out", and the "EDL In" and "EDL Out" the "Destination In" and "Destination Out", thus I don't uderstand why are you considering the "Transition Start" as a frame from the "Cut" fields, wouldn't be more clear to represent the transition with overlapping frames in your EDL In and Out fields? and that defines the transition start frame, end frame and duration by itself, and for completeness there could be the Transition Duration field just to avoid the arithmetic operation.

    And If there's a "Transition to" field maybe a "Transition from" would be a good idea too.

    Doc

  • 0
    Avatar
    Jean-Brice Royer

    I'd like to know where're you at on this subject. As far as I know, neither solutions have been implemented yet. At Mikros image, we implemented something much alike the first approach.

  • 0
    Avatar
    Jack James

    I'd like to chime in here and say that the correct terminology should be "edit" or "EDL", made up of "events" (or "edit events"). You also need to make a distinction between "source" timecodes and "record" (or less-commonly, "program") timecodes. You can use frame numbers interchangeably with timecodes. "Cut order" should be "event number". There really must be a transition type, even if it's just limited to "cut" and "dissolve" for now.

    I'd also make a case for having a third entity type in here too: "tracks", that sits in-between the two.

  • 0
    Avatar
    KP

    I just wanted to let you know we haven’t stopped thinking about this! We’ve had a lot of other things going on recently as you know, but this conversation has continued internally and here in our forums. We’ve been getting a lot of input from the community and that is nothing short of awesome. That said, we don’t have any specific updates to share with you right now, sorry. But we’re actively looking at the schema requirements for Shotgun itself, and compiling your input for our design meetings. Please continue to share your input in this thread.

    We hope to have more info for you soon!

  • 1
    Avatar
    Hugh Macdonald

    Hey all,

    I was just wondering where this had all got to? I missed a lot of the conversations over the last year or so, but it's just come up again internally that we would really like to be in a place where we'd like to be able to support cuts, especially as part of an RV review session.

     

    Personally, I'm all for keeping them separate from (although similar to) Playlists - I could see things getting confused between the two if they were just different settings on the same entity.

     

    Anyway, it'd be really interesting to know where this is, and what the plan for it is!

     

    Cheers

    Hugh

  • 0
    Avatar
    Don Parker

    Hey all, we're in design mode on all this now and will be ready to pitch our plans to you this summer (aiming for Siggraph).  We'll report back!

  • 0
    Avatar
    Armando Ricalde

    YES! Finally

    Thanks Don

    Cheers,

    Doc

  • 0
    Avatar
    Hugh Macdonald

    Ditto! Looking forward to it!

  • 0
    Avatar
    Marijn Eken

    Great that you're looking into this now. Would love to share thoughts once you have a draft.

  • 0
    Avatar
    Don Parker

    Yep, we plan to do a presentation to y'all for review/input/debate.  We're basically previs'ing it out for you.  Please stand by!

  • 0
    Avatar
    Mark Visser

    Any progress on this? We're at the point where we're going to roll our own entities just so we can support editorial in our pipeline, but having official support and integration with screening room would be great.

  • 0
    Avatar
    Ryan Mayeda

    Hi Mark.

    Sorry for spacing on your question.  This is in active development now and we showed the (early) designs back at Siggraph, and further iterations since.  I think we have another webinar coming up that will have more details, let me check on the date for that.  Alternately, we can schedule a quick call to discuss.

    Ryan.

Please sign in to leave a comment.