Publishing your work

Publish

Provides UI and functionality to publish files to Shotgun.


By: Shotgun Software
Latest Version: v0.10.4 (prod)
For our version numbers, we follow the Semantic Versioning standard.
System Name: tk-multi-publish

Overview

This document describes functionality only available if you have taken control over a Toolkit configuration. Please refer to the Shotgun Integrations User Guide for details. Multi Publish is a workflow app that artists can use to publish files to Shotgun. The app provides a simple UI on top of a highly customizable core that can be easily configured to publish any combination of files or data from a work scene and it's dependencies (e.g. the Maya scene, Nuke Script, OBJ's, Alembic Caches, Cameras, Textures, etc.). The multi publish app is used in all default configurations and can be easily configured to support workflows ranging from Alembic Cache generation in Maya to render publish in Nuke, art reference management in Photoshop or plate ingestion in the Shotgun and shell environments.

Documentation

The main interface is split into three key sections:

  • Primary Output. Items to be published are organized by their output type with the primary output given prominence at the top of the UI. There must be exactly one item to publish for the primary output which would typically be the scene file itself.

  • Secondary Outputs. Secondary outputs are optional and there can be as many as required. Items from the scene (e.g. a mesh in Maya) can also be associated with multiple secondary outputs (e.g. OBJ export together with an Alembic Cache export) if needed.

  • Publish Details. In addition to selecting the items to publish, the interface also allows the artist to choose the Shotgun task (if they haven't already specified this in the current Work Area), capture a thumbnail and add a comment.

When using the app, the artist workflow would typically be:

  1. Launch the app, select the items to be published, choose the Shotgun task, set the thumbnail, enter a comment and then click 'Publish'.
  2. The app will perform a pre-publish validation pass on the selected items.
  3. If there are any problems it will ask if it should continue. If it shouldn't then it will stop and display the problems, allowing them to be fixed before continuing.
  4. If there were no problems or it was told to ignore any problems, it will then publish all selected items, starting with the primary.
  5. As long as the primary item is successfully published, the app will finish by running a post-publish step. Typically this is used to version-up the current scene and do any cleanup.
  6. Finally the result: Success or Failure, is displayed together with any problems reported by the publish step.

Customizing Multi Publish

Multi Publish has been designed to be extremely configurable with all the key steps being implemented as a series of hooks. As well as separating out all engine specific calls from the app, making it easy to configure for new engines, these hooks also provide complete control over the publish process.

These are the main hooks that together comprise the publish workflow:

  • hook_scan_scene is responsible for scanning the current scene for items to be published. It should return a primary item together with any number of secondary items.
  • hook_primary_pre_publish is typically where you would validate that the primary item can be published. This can return a list of non-fatal problems that won't stop the publish.
  • hook_secondary_pre_publish is where the validation of all secondary items happens. Again this can return a list of problems that the user will have the opportunity to fix before continuing.
  • hook_primary_publish will publish the primary item for the primary output. Typically this is responsible for copying the current scene file to the publish area and registering it with Shotgun. It should return the full path to the published file which is then passed as a dependency to the secondary publish hook. Raising an exception within this hook will stop the publish and notify the user.
  • hook_secondary_publish publishes the secondary items for the secondary outputs. Again this typically exports or copies the published files and then registers them with Shotgun.
  • hook_post_publish is the final hook in the publish workflow allowing any clean up to be done at the end of a publish. In the default hooks, this is where the scene is versioned up. This hook will only by run if the primary publish succeeded without raising an exception.

These additional hooks can be used to customize the behaviour of the app.

  • hook_thumbnail can be used to provide a thumbnail for the publish in cases where performing a screen capture doesn't make any sense. If this returns a file path to a thumbnail then the screen capture functionality is disabled.
  • hook_copy_file is used to do any disk based file copying of files by the app. The default hooks use a utility method on the app which in turn calls this hook. To do this, call self.parent.copy_file(...) from within the hook.

Outputs & Items

It's important to understand the relationship between the outputs defined in the configuration and the items returned by the scan-scene hook.

Each output has a scene_item_type property which is an arbitrary string defined by the configuration.

The primary_scene_item_type must be unique. Multiple secondary outputs can share the same scene_item_type. For example, if the two secondary outputs: 'OBJ' and 'Alembic Cache' are given the same scene_item_type geometry, this indicates the intention that all geometry items from the scene should have both an OBJ and Alembic Cache file published for them.

Each item returned by the scan-scene hook has a type which must match a scene_item_type for one of the outputs defined in the configuration.

The app will combine the list of items with the list of outputs using the mapping between the output's scene_item_type and the item's type and present the final list to the user to allow them to select the items for each output to be published.

Related Apps and Documents

Scene Snapshot

This document describes functionality only available if you have taken control over a Toolkit configuration. Please refer to the Shotgun Integrations User Guide for details. A Shotgun Snapshot is a quick incremental backup that let's you version and manage increments of your work without sharing it with anyone else. Take a Snapshot, add a description and a thumbnail, and you create a point in time to which you can always go back to at a later point and restore. This is useful if you are making big changes and want to make sure you have a backup of previous versions of your scene.

Shotgun Workfiles

This document describes functionality only available if you have taken control over a Toolkit configuration. Please refer to the Shotgun Integrations User Guide for details. This application forms the basis for file management in the Shotgun Pipeline Toolkit. It lets you jump around quickly between your various Shotgun entities and gets you started working quickly. No path needs to be specified as the application manages that behind the scenes. The application helps you manage your working files inside a Work Area and makes it easy to share your work with others.

Write Node

This document describes functionality only available if you have taken control over a Toolkit configuration. Please refer to the Shotgun Integrations User Guide for details. This app contains a custom Write Node gizmo for Nuke, abstracting away the file system paths from the user, allowing them to focus on just naming the particular output. Shotgun takes care of the rest! This app is typically used in conjunction with the Publish app and if you install the publish app for nuke, you most likely want to install this one too!

Publishing Alembic from Maya

This tutorial shows how you can configure Multi Publish to export Alembic files when you are publishing groups of geometry from Maya.

Toolkit Pipeline Tutorial

This document describes functionality only available if you have taken control over a Toolkit configuration. Please refer to the Shotgun Integrations User Guide for details. This document provides a step-by-step tutorial for Technical Directors (TDs) setting up a simple end-to-end pipeline using Toolkit in an advanced way. New Toolkit users who finish this tutorial will understand the fundamental aspects of the platform as well as how to customize it based on their studio's specific needs. The tutorial also provides an introduction to the out-of-the-box Toolkit apps and how they fit together in the artist workflow.

Installation and Updates

Adding this App to the Shotgun Pipeline Toolkit

If you want to add this app to Project XYZ, in an environment named asset, execute the following command:

> tank Project XYZ install_app asset tk-maya tk-multi-publish

Updating to the latest version

If you already have this item installed in a project and you want to get the latest version, you can run the update command. You can either navigate to the tank command that comes with that specific project, and run it there:

> cd /my_tank_configs/project_xyz
> ./tank updates

Alternatively, you can run your studio tank command and specify the project name to tell it which project to run the update check for:

> tank Project XYZ updates

Collaboration and Evolution

If you have access to the Shotgun Pipeline Toolkit, you also have access to the source code for all apps, engines and frameworks in Github where we store and manage them. Feel free to evolve these items; use them as a base for further independent development, make changes (and submit pull requests back to us!) or simply tinker with them to see how they have been built and how the toolkit works. You can access this code repository at https://github.com/shotgunsoftware/tk-multi-publish.

Special Requirements

  • You need Shotgun Pipeline Toolkit Core API version v0.14.58 or higher to use this.

Configuration

Below is a summary of all the configuration settings used. These settings need to be defined in the environment file where you want to enable this App or Engine.

hook_primary_publish

Type: hook

Default Value: primary_publish

Description: Specify the hook that will actually do the publish of the primary task. This hook is passed the primary task that should be published. The hook is responsible for the entire publish process and should ensure to register the publish with Shotgun.

hook_post_publish

Type: hook

Default Value: post_publish

Description: Specify the hook that will be used to do any post-publish work. Typically, this hook will be responsible for versioning up the scene to the latest version.

display_name

Type: str

Default Value: Publish

Description: Specify the name that should be used in menus and the main publish dialog

primary_display_name

Type: str

Default Value: Current Work File

Description: This is the name the primary output will be given in the UI

secondary_outputs

Type: list

Description: No description available!

primary_scene_item_type

Type: str

Default Value: work_file

Description: This is the type used to match up items returned from the scan scene hook with with the primary output

allow_taskless_publishes

Type: bool

Default Value: True

Description: Allow publishing when no Task is specified. The publish will just be linked to the entity from the current work area

hook_copy_file

Type: hook

Default Value: copy_file

Description: Specify a hook to copy the file 'source_path' to 'target_path'. This hook is used in the 'copy_file' utility function and can be accessed from other hooks by calling self.parent.copy_file(source_path, target_path, task) from within the hook.

primary_publish_template

Type: template

Description: Template used to locate the primary published file within the file system. If None then this must be determined within the publish hook.

primary_tank_type

Type: tank_type

Description: This is the type that will be used when registering the primary published file with Shotgun. If not set then this must be determined within the publish hook.

template_work

Type: template

Description: A reference to a template which locates a work file on disk.

hook_primary_pre_publish

Type: hook

Default Value: primary_pre_publish

Description: Specify the hook that will run before the primary publish. This hook can be used to perform any validaiton on the primary task that is passed in to ensure that it is ready to be published.

hook_scan_scene

Type: hook

Default Value: scan_scene_{engine_name}

Description: Specify a hook to scan for items to publish. The hook should return a list dictionaries that represent the items to be published.

primary_icon

Type: config_path

Description: This is the icon used for the primary output in the UI

expand_single_items

Type: bool

Description: Define if an output should be expanable to show the items it contains when there is only a single item.

hook_thumbnail

Type: hook

Default Value: thumbnail

Description: Specify a hook to pre-generate a thumbnail for the publish.

hook_secondary_publish

Type: hook

Default Value: secondary_publish_{engine_name}

Description: Specify the hook that will be used to do the publish. This hook is passed a list of secondary tasks that are to be published. The hook is responsible for the entire publish process and should ensure it registers the published files with Shotgun

hook_secondary_pre_publish

Type: hook

Default Value: secondary_pre_publish_{engine_name}

Description: Specify the hook that will run before the publish. This hook can be used to perform any validaiton on the secondary tasks that are passed in to ensure that they are ready to be published.

primary_description

Type: str

Default Value: Publish and version up the current work file

Description: This is the description of the primary output used in the UI

Release Notes

Welcome to the release notes for this App. Below you will find an overview of all the changes we did for each release. We try to be as detailed as possible and include all bugs we have fixed, features we have added and things that may have changed. If you have questions about a particular release, don't hesitate to contact our support team!

v0.10.4

2017-Mar-17

Better handling of unicode file paths during pre-publish validation.

v0.10.3

2017-Mar-16

Switches to writing jpg for thumbnails instead of png.

Details:

This stops further libpng errors mentioned in v0.10.2 release notes from occurring.

v0.10.2

2017-Mar-14

Removes problematic profiles from png files used as icons.

Details:

This will silence libpng warnings from Qt when the png files are loaded as pixmaps. This is a second attempt as this profile stripping, and uses ImageMagick's "mogrify" command to achieve the desired result, which maintains color consistency.

v0.10.0

2017-Jan-30

Adds support for tk-photoshopcc.

v0.9.6

2016-Nov-16

[minor] UI layout tweaks to address further visual quirks discovered during v0.9.5 testing.

v0.9.5

2016-Nov-15

[minor] UI layout tweak to address visual quirk in Maya 2017 on Windows.

v0.9.4

2016-Aug-11

[minor] Removes log_error call that was in place for debugging purposes.

v0.9.3

2016-Aug-11

Properly supports changing contexts on the fly.

v0.9.2

2016-Jun-01

Changed default icons.

v0.9.1

2016-Mar-30

QA Fix: Verifies that 3ds Max writes the .abc file to disk before publishing.

v0.9.0

2016-Mar-29

Adds support for Alembic cache publishing for 3ds Max 2016+.

v0.8.7

2016-Feb-01

QA fix to use file base name as part of publish name

v0.8.6

2016-Feb-01

QA fix to avoid multiple path matches in output file matching

v0.8.5

2016-Jan-29

QA fixes for houdini publish hooks on Windows

v0.8.4

2016-Jan-22

Prevents annoying save popup for Houdini

v0.8.3

2016-Jan-22

Adds the ability to publish alembic/mantra outputs

v0.8.2

2016-Jan-21

Bug fix for Nuke Studio support in the tk-nuke scene operation hook.

v0.8.1

2016-Jan-19

Hotfix. Bug in conditional in primary publish.

v0.8.0

Adds Nuke Studio support.

v0.7.4

Adds user_data argument to hooks for sharing a dict of data between them.

Details:

The user_data dict is passed to all hooks when they are executed. This dict is shared across all hook executions and, as such, can be used to store data in within a hook that can then be accessed from subsequently run hooks. Because this is a new argument being passed to each of the hooks, it's possible that custom hooks will need to be updated to accept the additional argument. That being said, tk-multi-publish hooks are all setup to accept **kwargs out of the box, so unless that has been explicitly removed from a custom hook there will not be any problems.

v0.7.3

2015-Oct-29

Fix for QLayout warnings on app launch

v0.7.2

Merges Hiero hook operations into Nuke hooks.

Details:

Changes necessary to allow the tk-nuke engine to work within the Hiero mode of Nuke 9.0+.

v0.6.10

Minor fix for a publish item UI error in 3ds Max 2014

v0.6.9

throw exception if sending to Screening Room while in proxy mode and misc cleanup

Details:

Publishing proxy renders is not supported, this adds additional checks to ensure nothing slips by while in proxy mode in Nuke removed incorrectly named hook files for 3dsmaxplus engine fixed missing import

v0.6.8

Fix to unicode strings causing Houdini operations to fail.

v0.6.7

Added handling for edge-case crash when publishing channels from Mari

Details:

  • This change implements a workaround for a rare bug where publishing flattened channels that contain only a single layer could crash Mari.

v0.6.6

Adds .psb support in Photoshop.

Details:

This release adds Photoshop PSB support. In order to save your Photoshop scenes as PSBs, simply update your templates.yml file entries for Photoshop (photoshop_shot_* and photoshop_asset_*) to use the extension .psb. Note that you won't be able to use .psd and .psb at the same time for the same entity type. Also, you need the tk-photoshop engine's version to be greater or equal to v0.3.0. Finally, you also need tk-multi-workfiles to be updated to v0.6.7.

v0.6.5

Updated Nuke secondary publish hook to use new colorspace friendly method when creating movies for review

Details:

  • Previously, the colorspace was not being passed through from the Write node so if it was set to a non-default value, the rendered movie would be wrong.

v0.6.4

Added support for the new 3DS Max Engine supporting MaxPlus and Max version 2015.

v0.6.3

Improvements to the default Mari publish hooks

Details:

  • Fixed logic to find the next version to use for the published channels
  • Mari project is now saved when publish starts
  • Folders are now created before files are published

v0.6.2

Fixed broken publish paths and names for Mari texture publishes

v0.6.1

Fixed issue with Mari publish permanently flattening channels!

v0.6.0

Added support for Mari

Details:

  • Added Alembic publish as a Maya secondary publish
  • Added publish hooks for Mari to export texture channels and channel layers
  • Items returned from the scan-scene hook that don't match a secondary publish type defined in the configuration are now ignored. Previously an exception would have been raised.

v0.5.1

Added Motion Builder support

v0.5.0

Updated to use the new framework versioning syntax

Details:

  • This also requires core v0.14.58

v0.4.4

fix error in dependency check in Nuke when read nodes have empty filenames

Details: If read nodes in the current script have their file knob set to "" by a script, this will return None when reading the file knob value back which causes an error when trying to replace the path separator. This fix checks to make sure we have an actual (non-empty) file name before continuing on to try and match it with a template. If the file_name is an empty string or None, we skip the check and move on error free.

v0.4.3

Primary pre-publish errors are now displayed in the UI.

Details: - Previously, the user was informed that there were errors but the actual errors were never displayed - this has been fixed!

v0.4.2

Fixed issue when allow taskless publishes was set to False

Details: - Also changed the default back to True for this setting

v0.4.1

Various fixes and new features:

Details: - Added a new setting to allow 'No Task' selection for publishes when the current context doesn't specify a task and the pipeline supports task-less publishes. - Added a second per-item progress bar for publishes when there are more than one item being published. - Fixed some issues with the progress reporting where the progress bar wasn't behaving correctly. - Updated hook to use the latest interface.

v0.4.0

Changes to support Perforce workflow - version is now an optional parameter in all templates - added primary task parameter to secondary publish hooks - added primary and secondary tasks to post publish hook

v0.3.20

Updated Perforce implementation (limited release)

v0.3.19

Updated Perforce implementation (limited release)

v0.3.18.1

Updated Perforce implementation (limited release)

v0.3.18

Removed force-reset of Shotgun Write node render paths in the post-publish hook as it's no longer required for tk-nuke-writenode v1.0.0+

v0.3.17.1

Updated Perforce implementation (limited release)

v0.3.17

Fix for PyQt 4.6 compatibility.

Details: Fix for PyQt 4.6 compatibility.

v0.3.16.1

Updated Perforce implementation (limited release)

v0.3.16

Addition of setting to allow expanding of output items when there is only a single item

v0.3.15.1

Initial Perforce implementation (limited release)

v0.3.15

Order of Secondary outputs and items in the UI now respects the order from the configuration and scan-scene hook respecively

v0.3.14

Fixes for ordering of Secondary outputs

Details: - Order of outputs in the UI now respects the order of the secondary outputs in the configuration - The order of tasks sent to the secondary output hooks now matches the configured order of the secondary outputs - Updated default secondary publish hook for Nuke to ensure that renders are published before review versions

v0.3.13

Fixed Nuke secondary publish so that it doesn't rely on the order of tasks

v0.3.12

Fixed Python 2.5 compatibility issues

v0.3.11

Fixed issue when using non-ascii characters in the publish comment

v0.3.10

Tweaked the display name when name is not specified in the template

v0.3.9

Updated behaviour when name is either optional or absent from the templates

v0.3.8

Relaxed template validation in app manifest

v0.3.7

Updated to require widget framework v0.1.16

v0.3.6

Adjustments to the Photoshop Publish Support.

v0.3.5

Added Photoshop Support. Various Tweaks.

Details:

  • Added Photoshop Support in hooks.
  • Updated Autodesk icons to reflect new logos
  • fixed a minor path issue with houdini on windows
  • license updates
  • tweaks to softimage support

v0.3.4

Tweaked warning messages in default Nuke Hooks

v0.3.3

Improved Nuke Review Submission Functionality in Hooks.

v0.3.2

Minor changes to the nuke export hooks UI formatting.

v0.3.1

Added default Review Submission to Nuke publish hooks.

v0.3.0

Added Hiero and Houdini Support. Minor tweaks.

v0.2.10

Updated to use latest version of the widget framework

v0.2.9

Additional logging of call stacks for TankErrors raised within hooks

v0.2.8

Renames and support for the new name Sgtk.

v0.2.7

Updated to use the configured Published File Entity type.

v0.2.6

Added missing dependency analysis for maya scene primary publish

v0.2.5

Fixed a spelling mistake in a configuration file.

v0.2.4

Fixed app manifest to correctly allow a null value to be used as the secondary_output.publish_template setting

v0.2.3

Fixed various issues stopping Publish app from working with PyQt

v0.2.2

Various bug fixes and minor improvements.

Details:

  • Now uses generic tank QT paths, allowing it to work both in pyside and pyqt
  • Better display of task header when the task is provided via the context
  • added additional metadata to command handling
  • added a check in the nuke write node hook to ensure render path is not locked

v0.2.1

Configuration changes - the primary output is no longer stored in a dictionary.

v0.2.0

First non-preview release to the Tank Store.

v0.1.7

Initial Release to the Tank Store.

Follow

8 Comments

  • 0
    Avatar
    Julien Dubuisson

    Hey guys,

    I have a few template strings that I use in my scan/pre-publish hooks to search for specific item names in the scene or alternate publish paths

    and I'm manually checking in the scan hook if they are available in the config.

    What about having an entry in the publish config to add other templates than the primary and secondary publish templates? Those will be checked when starting the app and the app won't show up in the menu if something is missing.

    Is there someone else doing/needing that or am I doing something wrong with my templates?

    Cheers

    Julien

     

  • 0
    Avatar
    Alan Dann

    Hi Julien,

    You're not doing anything wrong and this is indeed a missing piece of the puzzle!  I agree that it would be great to be able to specify the templates that are used in the hooks (outside of the ones in the configuration) so that they can be validated at startup of the app/engine/etc.

    This could be part of the configuration but I'm also thinking that maybe this could be part of the hook - so you'd have a Hook base method called register_template(...) or something.  That way it would be in one place...

    I'll add this to the list to be discussed internally and add a feature ticket if the others agree.

    Out of curiosity, is not having this causing you problems or is it just a nice-to-have to help avoid future problems?

    Thanks

    Alan

  • 0
    Avatar
    Julien Dubuisson

    Hi Alan,

    I like the register_template idea in the hook class. Will this trigger a check at the app initialization or only when you call the hook? I would prefer the former as it would tell me I'm missing the templates before trying to run and fail any publish.

    It used to cause me errors with the old Tank (pre v0.13 ) when I was jumping from project to project with different configs, hooks and apps. But it was mainly bad version tracking on our side. It's not the case now because I'm properly checking for the templates in my scan hook before anything.

    Thanks

    Ju

  • 0
    Avatar
    Alan Dann

    Hi Julien,

    After discussion we agreed that being able to add the templates to the configuration (with the hook) would be best as this would allow validation without running hook code.

    I've created an internal ticket for this and we'll try to get to it soon.

    Thanks

    Alan

  • 0
    Avatar
    Fabian Dittrich

    Hi!

     

    There's a bug in tk-multi-publish v0.6.8, in

    tk-multi-publish\v0.6.8\python\tk_multi_publish\item.py

    is missing a

    from sgtk import TankError

    Cheers,

    Sebastian H

  • 0
    Avatar
    Alan Dann

    Thanks Sebastian :)

    I've added it in the code and it'll go out with the next release.

    Thanks again

    Alan

  • -2
    Avatar
    UndoLock

    How Can I ** fix error 53 on iPhone 6** using this tool? Can you help me please.

  • 1
    Avatar
    David van Heeswijk

    Hey guys,

    Is it possible for secondary publishes to have more than one level to expand to in the ui? Basicaly like a tree view, like in the Loader.

    Example, In Maya I have a group of object called "Table_Group". I like to be able to show the group in the publish ui and then drill down into it and only publish the "Desk" as a Alembic file. I don't want to just list all objects, since this clutters the ui quickly.

    Cheers,

    David

Please sign in to leave a comment.