Integrations Admin Guide

Table of Contents:

Introduction

Configuring software launches

      Example: Grouping versions of the same application, auto-detect

      Example: Grouping versions of the same application, manual mode

      Example: Restrict by users or groups

      Example: Restrict by project

      Example: Add your own Software

Configuring published file path resolution

      Resolving local file links

      Resolving file URLs

      Advanced configuration

Proxy configuration

Setting up your own configuration

            Launching the setup wizard from Desktop

            Select a configuration type

            Default configuration templates

            Basing your new project on an existing project

            Using a configuration template from git

            Browsing for a configuration template

            Setting up a storage

            Choosing a project folder name

            Selecting a configuration location

            What can I do once I have a configuration?

Advanced functionality

      Silent installs

Introduction

This document serves as a guide for administrators of Shotgun integrations. It's one of three: user, admin, and developer. Our User Guide is intended for artists who will be the end users of Shotgun integrations in their daily workflow, and our Developer Guide is technical documentation for those writing Python code to extend the functionality. This document falls between those two: it's intended for those who are implementing Shotgun integrations for a studio, managing software versions, and making storage decisions for published files.

Configuring software launches

It’s simple to rely on Shotgun’s auto-detection of host applications on your system: just launch Shotgun Desktop, choose a project, and Desktop will display launchers for all supported software packages that it finds in standard application directories. But we also offer robust tools for more fine-grained management of the software in your studio. You can restrict application visibility to specific projects, groups, or even individual users. You can specify Versions, deactivate a given software package across your site, and group software packages together. All of this is managed through Software entities in Shotgun.

When you create a new Shotgun site, it will have a set of default Software entities—one for each supported host application. You can modify these and add your own to manage the software that shows up in Desktop exactly how you want it.

To see your Software entities in Shotgun, open the Admin menu by clicking on the profile icon in the upper right corner of the screen, and choose Software.

The Software entity has the following fields:

  • Software Name: The display name of the Software in Desktop.
  • Thumbnail: Uploaded image file for Desktop icon.
  • Status: Controls whether or not the Software is available.
  • Engine: The name of the integration for the content creation tool.
  • Products: For Software packages that include variants (e.g., Houdini vs. Houdini FX), you can specify a comma separated list here. Valid only in auto-detect mode, not manual mode.
  • Versions: Specific versions of the software to display. You can specify a comma separated list here. Valid only in auto-detect mode, not manual mode.
  • Group: Entities with the same value for the Group field will be grouped under a single icon in Desktop and a single menu in Shotgun. For example, you could create an FX group that includes Houdini and Nuke.
  • Group Default: When one member of a group has Group Default checked, clicking the icon or menu item for the group will launch this software.
  • Projects: A way to restrict software to certain projects.
  • User Restrictions: A way to restrict software to certain users or groups.
  • Linux/Mac/Windows Path: Use these fields to explicitly specify an OS-specific path to software.
  • Linux/Mac/Windows Args: Commandline args to append to the command when launching the Software.

We can learn a lot about how these fields work together by demonstrating some ways of using them.

Example: Grouping versions of the same application, auto-detect

Say you have three versions of Maya on your filesystem: Maya 2016, Maya 2017, and Maya 2018. You want to make all of these available to your artists, but you want them to be grouped under a single icon in Desktop.

If these three versions of Maya are installed in the standard location on your filesystem, then this will all happen automatically. When you select a project in Desktop, it will scan the standard applications directory on the local machine, and will find the three versions. Since you already have a Maya software entity in Shotgun, with no specific versions or paths specified, it will display all versions it finds in Desktop.

A few things to note here:

  • When Shotgun auto-detects your software, a single Software entity generates the menu items for all versions.
  • None of the Path fields have values specified. The Software entity is in auto-detect mode, so the App is assumed to be in the standard location.

These will show up in Desktop as you see here: one icon for Maya, with a drop-down listing all the available versions. If you click on the icon itself, you’ll launch the latest version of Maya.

Example: Grouping versions of the same application, manual mode

It’s perfectly fine to store Maya in a non-standard location in your studio. You’ll just need to create your own Software entities, and specify paths to let Shotgun know where to find your software. Your setup may look like this:

Some notes here:

  • Unlike in auto-detect mode, here you have a Software entity for each version of a given software package.
  • In order to group them together, use the Group and Group Default fields. Software entities that share the same value for Group will be grouped in Desktop in a dropdown under a single icon, which uses the Group value as its name.
  • When you click on that icon itself, you’ll launch the software within the group with Group Default checked.
  • When you specify a value for any of Linux Path, Mac Path, or Windows Path on a Software entity, that entity will shift to Manual mode. Unlike auto-detect mode, where the software would show up in Desktop when a path field is empty, in manual mode, a software package will only show up on a given operating system if a path is specified for it and the file exists at the specified path.
  • In this example, none of the three Maya versions would show up in Desktop on Windows because no Windows Path has been specified.

Example: Restrict by users or groups

Now, say with that last example that we’re not ready to make Maya 2018 available to all users just yet. But we do want TDs, Devs, and our QA engineer, Tessa Tester, to be able to access it. We can achieve this with the User Restrictions field. Here’s an example:

We made a couple changes from the last example:

  • The group default is now Maya 2017. We want that to be the production version, so with that box checked, clicking the icon for Maya will now go to this version.
  • We’ve added a few values to the User Restrictions field: It can take both users and groups, and we’ve added our Dev and TD groups, as well as the user Tessa Tester. Now, only those people will see Maya 2018 in Desktop.

Example: Restrict by project

Sometimes you want to do more complex version management across projects in your studio. You may have a project in a crunch to deliver, which you want to lock off from new versions of software, while at the same time, its sequel may just be starting up and able to evaluate newer versions. In this case, you may have your Software entities set up like this:

A few important things to note:

  • We’ve removed the Group and Group Default values here, as only one Maya version will ever show up in Desktop for a given environment.
  • We’ve set the Software Name for all three versions to “Maya”. This way, on every project, users will have an icon with the same name, but it will point to different versions depending on what’s configured here.
  • We’ve set Maya 2016’s Status field to Disabled. We are no longer using this version in our studio, and this field toggles global visibility across all projects.
  • We’ve specified values for Projects for Maya 2017 and Maya 2018. This Projects field acts as a restriction. Maya 2017 will only show up in the Chicken Planet project, and Maya 2018 will only show up in Chicken Planet II.
  • Note that once you’ve specified a value for Projects for a Software entity, that Software will only show up in the projects you've specified. So, if you have other projects in your studio in addition to the Chicken Planet series, you’ll need to specify software for them explicitly.

Example: Add your own Software

There are several reasons you might add a new software entity in addition to those that Shotgun Desktop has auto-detected on your system:

  • You want to make an application for which there is no engine available to your users through Desktop.
  • You have in-house software, or third-party software that we don’t have an integration for, for which you’ve written your own engine.
  • Your software doesn’t live in a standard location, so you want to point Shotgun to it manually. (This case was described in the “Grouping versions of the same Application, Manual mode” example above.)

In these cases, you can add your own Software entities. You'll need to have a value for the Software Name field. If you're using an in-house engine for your software, specify the engine name in the Engine field. Some studios may want to include apps in Desktop that don’t have Shotgun integrations, as a convenience for artists. Your artists can launch the app straight from Desktop. You can even use all of the settings above to manage versions and usage restrictions. In this case, leave the Engine field empty, but you'll need to specify a value for at least one of Mac Path, Linux Path, and Windows Path.

Configuring published file path resolution

When you publish a file, the Publisher creates a PublishedFile entity in Shotgun, which includes a File/Link field called Path. Later on, a different user may try to load this file into their own work session using the Loader. The Loader uses complex logic to resolve a valid local path to the PublishedFile across operating systems.

The way in which the Loader attempts to resolve the publish data into a path depends on whether the the publish is associated with a local file link or a file:// URL.

Resolving local file links

Local file links are generated automatically at publish time if the path you are publishing matches any local storage defined in the Shotgun Site Preferences. If the publish is a local file link, its local operating system representation will be used. Read more about local file links here.

If a local storage doesn’t define a path for the operating system you are currently using, you can use an environment variable to specify your local storage root. The name of the environment variable should take the form of SHOTGUN_PATH_<WINDOWS|MAC|LINUX>_<STORAGENAME>. So, if you wanted to define a path on a Mac for a storage root called "Renders", you'd create a SHOTGUN_PATH_MAC_RENDERS environment variable. Let's go deeper with that example:

  • Say your Shotgun site has a storage root called "Renders", with the following paths specified:
  • Linux path: /studio/renders/
  • Windows path: S:\renders\
  • Mac path: <blank>

  • You are on a Mac.

  • You want to load a publish with the path /studio/renders/sq100/sh001/bg/bg.001.exr into your session.

The Loader can parse the path and deduce that /studio/renders/ is the storage root part of it, but no storage root is defined for Mac. So, it will look for a SHOTGUN_PATH_MAC_RENDERS environment variable, and if it finds one, it will replace /studio/renders in the path with its value.

Note: If you define a SHOTGUN_PATH_MAC_RENDERS environment variable, and the local storage Renders does have Mac path set, the local storage value will be used and a warning will be logged.

Note: If no storage can be resolved for the current operating system, a PublishPathNotDefinedError is raised.

Resolving file URLs

The Loader also supports the resolution of file:// URLs. At publish time, if the path you are publishing does not match any of your site's local storages, the path is saved as a file:// URL. Contrary to local file links, these paths are not stored in a multi-OS representation, but are just defined for the operating system where they were created.

If you are trying to resolve a file:// URL on a different operating system from the one where where the URL was created, the Loader will attempt to resolve it into a valid path using a series of approaches:

  • First, it will look for the three environment variables SHOTGUN_PATH_WINDOWS, SHOTGUN_PATH_MAC, and SHOTGUN_PATH_LINUX. If these are defined, the method will attempt to translate the path this way. For example, if you are trying to resolve file:///prod/proj_x/assets/bush/file.txt on Windows, you could set up SHOTGUN_PATH_WINDOWS=P:\prod and SHOTGUN_PATH_LINUX=/prod in order to hint the way the path should be resolved.
  • If you want to use more than one set of environment variables, in order to represent multiple storages, this is possible by extending the above variable name syntax with a suffix:
  • If you have a storage for renders, you could for example define SHOTGUN_PATH_LINUX_RENDERS, SHOTGUN_PATH_MAC_RENDERS, and SHOTGUN_PATH_WINDOWS_RENDERS in order to provide a translation mechanism for all file:// URLs published that refer to data inside your render storage.
  • Then, if you also have a storage for editorial data, you could define SHOTGUN_PATH_LINUX_EDITORIAL, SHOTGUN_PATH_MAC_EDITORIAL, and SHOTGUN_PATH_WINDOWS_EDITORIAL, in order to provide a translation mechanism for your editorial storage roots.

Once you have standardized on these environment variables, you could consider converting them into a Shotgun local storage. Once they are defined in the Shotgun preferences, they will be automatically picked up and no environment variables will be needed.

  • In addition to the above, all local storages defined in the Shotgun preferences will be handled the same way.
  • If a local storage has been defined, but an operating system is missing, this can be supplied via an environment variable. For example, if there is a local storage named Renders that is defined on Linux and Windows, you can extend to support mac by creating an environment variable named SHOTGUN_PATH_MAC_RENDERS. The general syntax for this is SHOTGUN_PATH_<WINDOWS|MAC|LINUX>_<STORAGENAME>.
  • If no root matches, the file path will be returned as is.

Here's an example:

Say you've published the file /projects/some/file.txt on Linux, and a Shotgun publish with the URL file:///projects/some/file.txt was generated. In your studio, the Linux path /projects equates to Q:\projects on Windows, and hence you expect the full path to be translated to Q:\projects\some\file.txt.

All of the following setups would handle this:

  • A general environment-based override:
  • SHOTGUN_PATH_LINUX=/projects
  • SHOTGUN_PATH_WINDOWS=Q:\projects
  • SHOTGUN_PATH_MAC=/projects

  • A Shotgun local storage called “Projects”, set up with:

  • Linux Path: /projects
  • Windows Path: Q:\projects
  • Mac Path: /projects

  • A Shotgun local storage called “Projects”, augmented with an environment variable:

  • Linux Path: /projects
  • Windows Path:`
  • Mac Path:/projects`
  • SHOTGUN_PATH_WINDOWS_PROJECTS=Q:\projects

Note: If you have a local storage Renders defined in Shotgun with Linux path set, and also a SHOTGUN_PATH_LINUX_RENDERS environment variable defined, the storage will take precedence, the environment variable will be ignored, and a warning will be logged. Generally speaking, local storage definitions always take precedence over environment variables.

Advanced configuration

For information on the underlying method that performs the resolution of PublishedFile paths, take a look at our developer reference docs.

If you are using Advanced Project Setup, you can add support beyond local file links and file:// URLs by customizing the resolve_publish core hook. Possible customizations include:

  • Publishes with associated uploaded files could be automatically downloaded into an appropriate cache location by the core hook and the path would be be returned.
  • Custom URL schemes (such as perforce://) could be resolved into local paths.

Proxy configuration

If your studio is using a proxy server, if you want to pre-populate the initial login screen with some values, or if you want to tweak how the browser based application launcher integrates with the Shotgun Desktop, there is a special configuration file called toolkit.ini.

The file is searched for in multiple locations and in the following order:

  1. An environment variable named SGTK_PREFERENCES_LOCATION that points to a file.
  2. An environment variable named SGTK_DESKTOP_CONFIG_LOCATION that points to a file. (deprecated)
  3. Inside the Shotgun Toolkit preferences file.
  4. Inside the Shotgun Desktop preferences file. (deprecated)

The Shotgun Toolkit preferences file is located at:

  • Windows: %APPDATA%\Shotgun\Preferences\toolkit.ini
  • macOS: ~/Library/Preferences/Shotgun/toolkit.ini
  • Linux: ~/.shotgun/preferences/toolkit.ini

The Shotgun Desktop preferences file is located at:

  • Windows: %APPDATA%\Shotgun\desktop\config\config.ini
  • macOS: ~/Library/Caches/Shotgun/desktop/config/config.ini
  • Linux: ~/shotgun/desktop/config/config.ini

If you wish to store your configuration file somewhere else on your computer or on your network, you can set the SGTK_PREFERENCES_LOCATION environment variable and then launch the Shotgun Desktop.

A fifth, but deprecated option is to store the configuration file inside the installation folder on Windows and Linux. On Mac OS, it is a bit tricky and the file needs to be added to the application bundle, in the following location:

Note that storing the configuration file inside the application bundle is not recommended on Mac OS. Adding a file in that location invalidates the package's digital signature. Such a package will be flagged as altered by Mac OS and security prompts will appear during the execution of the application, notably to allow communications to go through the firewall.

You can see a documented example of a configuration file here.

Please note that you can use environment variables as well as hard coded values in this file, so that you could, for example, pick up the default user name to suggest to a user via the USERNAME variable that exists on Windows.


Running Shotgun Desktop with a locally hosted site

If your Shotgun site URL does not end with shotgunstudio.com, it means that you are running a local Shotgun site. In this case, it is possible that your site has not yet been fully prepared for Shotgun integrations and the Shotgun team may need to go in and do some small adjustments before you can get going! In this case, please submit a ticket and we'll help sort you out.

Connecting to the app store with a locally hosted site

If you are using a local Shotgun site with access to the Internet through a proxy, you might want to set an HTTP proxy for accessing the app store, but not the local Shotgun website. To do this, simply add app_store_http_proxy: <proxy_server_address> to shotgun.yml, where <proxy_server_address> is a string that follows the convention documented in our developer docs.

Setting up your own configuration

This guide reviews the kind of admin possible with out-of-the-box configuration. This integration is built on top of Shotgun's Toolkit platform, which supports much richer customization. To go from the out of the box behavior to something you can configure, you need to use a wizard to create a configuration specific to a project.

The Shotgun Desktop project wizard. It will take you through a step by step process and at the end you have a fully working Toolkit project, ready to be used. Each section below explains in detail each of the steps of the wizard with examples and suggestions of sensible default values in case you are not sure how to set things up.

Launching the setup wizard from Desktop

Once you have navigated to a project there will be an "Advanced Project Setup..." menu item in the user menu in the bottom right hand of Desktop. If you click on this menu item you will launch a wizard that will walk you through the setup of the configuration for a project.

Select a configuration type

When you start configuring a new project, the first thing to decide is which configuration template to use. A configuration template is essentially the complete project configuration with all settings, file system templates, apps and logic needed to run the project.

  • If this is your very first project, head over to the Shotgun defaults to get you started.
  • If you already have configured projects and configurations for previous projects, you can easily reuse these by basing your new project on an existing project
  • For advanced workflows, you can use external configurations or configs stored in git repositories.

Default configuration templates

This is the place to go if you want to start from scratch. The various Shotgun default configurations contain all the latest apps and engines set up with a default file structure and file naming convention.

Typically, once you have installed the default configuration, you need to go in and tweak the config and adjust it to work with your pipeline, your application paths etc. Once you have got a project up and running, you can base your next project on this project, and keep going and evolving your studio configuration that way.

The Default Configuration

This is the default Toolkit VFX configuration and usually a great starting point when you start setting things up. It comes with Hiero, 3dsmax, Nuke, Maya, Mari, Houdini, and Motionbuilder set up and contains a simple, straight forward folder setup on disk.

This config is for demo purposes and is a good way to get an overview of what apps are available and how Toolkit can integrate with an existing pipeline.

The configuration contains a number of different pieces:

  • A file system setup
  • A set of templates to identify key locations on disk
  • A set of preconfigured engines and apps which are chained together into a workflow.

The file system setup is minimal, focused on providing a demo and overview rather than being a fully featured file system. We strongly suggest that you get in touch with the Toolkit team if you need assistance when setting up file and folder structures.

File System Overview

The standard config handles Assets and Shots in Shotgun. It breaks things down per Pipeline Step. A pipeline step is similar to a department. Each pipeline step contains work and publish areas for the various supported applications. The Shot structure looks like this:

Applications and workflows

The config contains the following components:

  • Maya, Nuke, 3dsmax, Hiero, Houdini, Photoshop, and Motionbuilder support
  • Shotgun Application Launchers
  • Publishing, Snapshotting, and Version Control
  • A Nuke custom Write Node
  • Shotgun integration
  • A number of minor tools and utilities

In addition to the apps above, you can easily install additional apps and engines once the config has been installed.

The Multi-Root Configuration

This configuration has been deprecated.

You can follow these instructions to split a project over different data roots in the filesystem.

The Games Configuration

This configuration has been deprecated.

You can get an overview of how our integration with perforce works in our Perforce tutorial.

You can follow these instructions to enable Perforce integration for a configuration. You can follow these instructions to change the directory structure to use different entities from Shotgun.

The Flame Configuration

This configuration has been deprecated. The functionality has been added to the default configuration.

Basing your new project on an existing project

This is a quick and convenient way to get up and running with a new project with all the defaults and settings that you had in a previous project. Toolkit will simply copy across the configuration from your old setup to the new project. This is a simple and pragmatic way to evolve your configuration - each new project is based on an older project.

For more ways and documentation on how to evolve and maintain your pipeline configuration, see here:

Using a configuration template from git

Use this option if you want to keep your project's configuration connected to source control. Specify a url to a remote git or github repository and the setup process will clone it for you. Note that this is not just github, but works with any git repository. Just make sure that the path to the repository ends with .git, and Toolkit will try to process it as a git setup. Because your project configuration is a git repository, you can commit and push any changes you make to your master repository and beyond that to other projects. Using a github based configuration makes it easy to keep multiple Toolkit projects in sync. You can read more about it here:

Please note that if you are running on Windows, you need to have git installed on your machine and accessible in your PATH. On Linux and Mac OS X, it is usually installed by default.

Browsing for a configuration template

Use this option if you have a configuration on disk, either as a folder or zipped up as a zip file. This can be useful if someone has emailed a configuration to you or if you keep a master config on disk which you are basing all your projects on. This is usually an expert option and we recommend either using a config from another project or one of our app store default configs.

Setting up a storage

Each Toolkit project writes all its files and data to one or more shared storage locations on disk. For example, a configuration may require one storage where it keeps textures, one where it keeps renders and one where it stores scene files. The main project storage is always called primary.

Normally, these storages are controlled from within the Shotgun Site Preferences, under the File Management tab. The Toolkit Setup wizard will check to make sure all the different storages that the configuration needs have been set up properly. If they are missing or don't have a path set for the current operating system, you will be prompted to enter values.

Once the project is being set up, Toolkit will create a folder for each new project in each of the storage locations. For example, if your primary storage location is /mnt/projects, a project called The Edwardian Cry would end up in /mnt/projects/the_edwardian_cry. And if the config is using more than just the primary storage, each of the storages would end up with an the_edwardian_cry folder.

Your primary storage location is typically something like /mnt/projects or \\studio\projects and is typically a location where you are already storing project data, grouped by projects. It is almost always on a shared network storage.

Choosing a project folder name

Now it is time to choose a disk name for your project. This folder will be created in all the different storages which are needed by the configuration. You can see a quick preview in the UI - for most configurations this will only preview the primary storage, but if you are using a multi root config, additional storages will show up too. Toolkit will suggest a default project name based on the name in Shotgun. Feel free to adjust it in order to create what is right for your setup.

Selecting a configuration location

Lastly, please decide where to put your configuration files on disk. Toolkit will suggest a location based on previous projects, so that they all end up in the same place on disk.

The configuration normally resides on a shared storage or disk, so that it can be accessed by all users in the studio who needs it. If you are planning on using more than one operating system for this project, make sure to enter all the necessary paths. All paths should represent the same location on disk. Often, the path can be the same on Mac OS X and Linux but will be different on Windows.

If this is your first project, you typically want to identify a shared area on disk where you store all your future pipeline configurations. This is typically a location where you store software or software settings shared across the studio. This could be something like /mnt/software/shotgun. It may vary depending on your studio network and file naming conventions.

When you set up your first configuration, set it up with paths for all the platforms you use in your studio. This will make it easier later on to create an environment which is accessible from all machines. As a hypothetical example, if your project name is Golden Circle you may type in the following three paths:

linux:   /mnt/software/shotgun/golden_circle
macosx:  /servers/production/software/shotgun/golden_circle
windows: \\prod\software\shotgun\golden_circle

What can I do once I have a configuration?

Once you are up and running with your first configuration, please navigate to our 'next steps' documentation to learn more about how to configure and adjust Toolkit to better suite your studio needs:

You can also learn more in our Advanced Project Setup documentation.

Advanced functionality

Silent installs

If you are on a Windows network, you can use the argument "/S" to force the .exe Shotgun Desktop installer to do a silent install. Then you can push a copy of the shortcut to the executable to the startup folder.

Tell the doc gen system that we don't want a header by adding this special markup: TANK_NO_HEADER

Follow

5 Comments

  • 0
    Avatar
    Rob Aitchison

    "It’s simple to rely on Shotgun’s auto-detection of host applications on your system: just launch Shotgun Desktop and choose a project, and Desktop will display launchers for all software packages that it finds in standard application directories, for which we support integration"

    Is this part ready?  I just created a new Project, ran the Integration Beta and got the Set Up Toolkit Project screen - is that expected?

  • 0
    Avatar
    Travis Mosley

    Is it possible to expose a hook for business logic of turning a software entity into a command when in "manual mode"?

  • 0
    Avatar
    Rob Blau

    Hey Rob, I think we worked through your question in a support ticket.  If you have taken over the site configuration, then you will still use that by default (so everything is backward compatible).  I believe that was the reason you saw this.  You'd need to either update your manual site configuration with the new releases to match behavior, or get rid of the site config to revert back to the latest out of the box behavior.

  • 0
    Avatar
    Rob Blau

    Hi Travis, you can create a Software entity and point it at a script if you wanted to, but there isn't a direct way to use that to run commands when you have taken over a config.  To get commands into the Shotgun menu in this case the process is the same as it has been, you need to add the app that will register the command to the tk-shotgun engine instance for the appropriate environment in your configuration.

  • 0
    Avatar
    Travis Mosley

    Thanks Rob. I'm looking for more control in how a software entity resolves what to launch in Shotgun Desktop, without writing my own launcher and everything. This seems precisely the kind of task hooks excel at. 

Please sign in to leave a comment.