Setting up a gateway server – Shotgun Support

This article describes how you can allow access to a hosted Shotgun site while meeting the internet restriction policies in place at your studio.

Scope and assumptions

This document is intended for the System and Shotgun Administrator.

It is assumed that the reader has basic knowledge about proxy, networking, and HTTP protocols.

Disclaimer

This documentation is provided as a guide to our clients, in order to help them set up a gateway server. While we can provide some help, gateway servers are the responsibility of clients and will not be implemented by Shotgun.

Additionally, we’ve linked to external sources in this documentation, so please use your own best judgement when referencing.

Using a gateway server to access Shotgun in the cloud

A gateway server allows users to have access to their hosted Shotgun site, without having access to the rest of the internet. It forwards filtered requests to Shotgun end-points, but blocks all the other requests. This means that when a user requests *.shotgunstudio.com, the request will go through. But if a user requests an unauthorized site, the request will not go through.

Why use a gateway server?

You want to isolate your network from the Internet, while still allowing your users to access your Shotgun Hosted instance;
You want to connect to Shotgun using deprecated connection/encryption protocols (DO THIS AT YOUR OWN RISK).

We recommend you use a gateway server instead of setting up a proxy server for Shotgun. With a gateway server:

You don’t have to deal with allowing the list of complex IP ranges that can change over time;
You can still benefit from all the Web Acceleration features included in each Shotgun Hosted Site;
You will see better performance with Amazon S3 compared to the S3 proxy solution;
You will see better performance with all playback and all uploads, compared to the S3 proxy solution.

Getting started

There are three main steps to put in place a gateway server.

Put in place a split horizon Domain Name System (DNS).
Set up the gateway server using HAProxy.
Configure the users environment to use your gateway.

Step 0: Getting ready

In order to correctly configure the gateway server, you will need the following information:

Your site name (<your_site_name>.shotgunstudio.com)
The name of your particular media bucket, which is based on your regional zone to optimize performance. See this article for information about how to get this information: https://support.shotgunsoftware.com/hc/en-us/articles/115000022554-Selecting-a-storage-location-for-uploaded-files-overview. If you have problems figuring out this information, please contact Shotgun Support.
Two distinct internal IP addresses are assigned to the gateway server: one to listen on <your_site_name>.shotgunstudio.com and one to listen on the S3 url. A third IP may be required if you are using Toolkit. The range of the IP addresses depends on how the network at your studio is set-up.

Step 1: Split horizon DNS

Split horizon DNS allows for giving a different answer to a query (in this case, URL resolving) depending on the source of the query. Your split horizon DNS would, for people in your studio, resolve your Shotgun URL to an internal IP address (the gateway server) instead of resolving to Shotgun Public IPs.

The gateway server will forward all requests to the Shotgun IPs. The gateway server can have unrestricted access to make requests out to the internet, but all Artists will see is the internal server, limiting access to Shotgun Internet Resources only.

Setting up a DNS in such a configuration requires some basic IT skills. Jensd is one example of how to set up the parent/child relationship, in Appendix 1 at the end of that article.

If you don't already have a DNS server, you will want to order or dedicate a server to the task. The example in this article are for Linux systems.

Step 2: Configure HAProxy as a Shotgun Gateway

For implementing the gateway server, we suggest you use HAProxy. HAProxy is a reliable, simple and high performance TCP/HTTP load balancer that would allow you to implement the gateway easily. We recommend using at least version 1.6. Hardware requirements for HAProxy can be found here.

You will find in the Appendix 2 an example of an HAProxy configuration file.

Step 3: Configure your users' environment

The steps below apply if you are using a self-signed certificate.

If your network configuration allows you to use a Trusted Certificate (you will need to contact a Trusted provider), we highly recommend you to do so.

Note: Using self-signed untrusted certificates is a security risk (See wikipedia.org/wiki/Self-signed_certificate#Security_issues) and should be used only if there are no alternatives.

For all users	Update your router or your user's machine to advertise the new DNS server.
For browser users	Add the certificate authority to your keychain or Chrome will complain about unsafe certificates
For Python API users	Add the certificate authority to your keychain or set SHOTGUN_API_CACERTS
For REST API users	Add the certificate authority to your keychain or pass it down to requests or the library you are using
Toolkit users	Add the certificate authority to your keychain or set SHOTGUN_API_CACERTS
Things to look into	toolkit.ini could have a shotgun_api_cacerts setting so there would be no need for an env var.

Additional things to look into:

toolkit.ini could have a shotgun_api_cacerts setting so there would be no need for an env var.

Follow up steps

IP access restriction by your Shotgun site

If you set up a gateway server, you probably want your Shotgun site to only accept requests coming from your studio IPs. See "IP Whitelisting" on Shotgun Help Center for more details.

FAQ

Will the gateway server complexify an Artist’s setup?

No, the gateway server will not change an Artist’s internet access. If Artists are currently allowed to browse certain sites in a restricted way, they will still be able to do so. From an Artist’s perspective, the change will be imperceptible.

All URLs will continue to work as before. The same is true for emails, and URLs in Shotgun-generated emails.

Will the gateway server affect the speed of my site?

No, you will see the same speeds as direct access. You may even see faster speeds due to connection caching by the gateway.

Can I monitor what is forwarded?

Yes, you can log all traffic routed through the forwarding agent. It is possible to monitor the activity from an external angle, and track which IPs are using Shotgun, how many requests are made, etc. However, because the traffic is encrypted over HTTPS, the content of the requests will be encrypted.

Will my other tools work with the gateway server?

Yes, all other tools will continue to work. This includes Toolkit, the Shotgun API, RV, etc.

Why does tank.shotgunstudio.com also need to be forwarded?

Tank.shotgunstudio.com is the Toolkit App Store. Toolkit can operate without it, but calls to that site must also be forwarded for Toolkit updates to succeed.

Appendix 1 - Split horizon DNS (Ubuntu)

The following steps have been done on Debian. Steps and location of configuration files might be different for different platforms.

sudo apt-get install bind9

Open /etc/bind/named.conf and configure named. Below is a minimal DNS configuration file for bind9.

options { 
    directory "/var/cache/bind"; 
    # tells the dns that zones it doesn't understand should be handled by another DNS 
    forwarders { 
        8.8.8.8; # Google's DNS. 
         //you can add here other DNS servers 
    }; 
    dnssec-validation auto; 
    dnssec-enable yes; 
    auth-nxdomain no;    # conform to RFC1035 
    listen-on-v6 { any; }; 
}; 
logging { 
    channel bind.query.log { 
        file "/var/log/dns.query.log" versions 600 size 30m; 
        print-time yes; 
        print-category yes; 
        print-severity yes; 
        severity debug 3; 
    }; 
    // IF YOU DO NOT HAVE SYSLOG, REMOVE THIS. 
    channel default_syslog { 
        print-time yes; 
        print-category yes; 
        print-severity yes; 
        syslog daemon; 
        severity info; 
    }; 
    channel default_log { 
        file "/var/named/log/default" versions 3 size 20m; 
        print-time yes; 
        print-category yes; 
        print-severity yes; 
        severity info; 
    }; 
    channel client_security_log { 
        file "/var/named/log/client_security" versions 3 size 20m; 
        print-time yes; 
        print-category yes; 
        print-severity yes; 
        severity info; 
    }; 
    // IF YOU DO NOT HAVE SYSLOG, PLEASE REMOVE `default_syslog;` FROM LINES BELLOW 
    category default  { default_syslog; default_log; }; 
    category config   { default_syslog; default_log; }; 
    category dispatch { default_syslog; default_log; }; 
    category network  { default_syslog; default_log; }; 
    category general  { default_syslog; default_log; }; 
    category client   { default_syslog; client_security_log; };        
    category security { default_syslog; client_security_log; }; 
    category queries { bind.query.log; }; 
}; 
# Defines a zone we want to answer DNS query for 
zone "shotgunstudio.com" { 
    type master; 
    file "/etc/bind/db.shotgunstudio.com"; 
};

Open /etc/bind/db.shotgunstudio.com and paste this minimal configuration file for the bind9 zone for shotgunstudio.com (replace mysite with your studio URL):

$TTL    1
# This is the SOA header. I lifted this from somewhere and put very small TTL
@       IN      SOA     ns.shotgunstudio.com. root.shotgunstudio.com. (
                  4     ; Serial
                  5     ; Refresh
                  5     ; Retry
                  5     ; Expire
                  5 )   ; Negative Cache TTL

@       IN              NS              ns.shotgunstudio.com.
ns.shotgunstudio.com.      IN      A       192.168.1.174 ;A Server IP...
# This is where we are mapping our internal site to our HAProxy server address
mysite.shotgunstudio.com.  IN      A       192.168.1.174 ;Another Server IP

Appendix 2 - Configure HAProxy as a gateway

Install haproxy. Docker users can use the latest version of the haproxy:alpine container: https://hub.docker.com/_/haproxy/

apt-get install haproxy

Open /etc/haproxy/haproxy.cfg and configure haproxy. Below is a configuration example for haproxy 1.6. Replace the part between <> with your own information, including the <> symbols.

global
 maxconn 4096
 pidfile /tmp/haproxy-queue.pid
  
defaults
 log global
 timeout connect 300000
 timeout client 300000
 timeout server 300000

# handle external ip changes
resolvers external_dns
 nameserver dns1 <external_dns_ip>:53
 resolve_retries 3
 timeout retry 1s
 hold valid 10s
 
# in case someone enters SG address without https
listen http_redirect 
 bind <internal_ip_address_1>:80
 mode http
 redirect scheme https code 301 if !{ ssl_fc }
 
listen shotgun_proxy
 bind <internal_ip_address_1>:443
 mode tcp
 server app1 <your_site_name>.shotgunstudio.com:443
 
listen shotgun_s3_proxy
 bind <internal_ip_address_2>:443
 mode tcp
 server app1 sg-media-usor-01.s3-accelerate.amazonaws.com:443

listen shotgun_tank_proxy
 bind <internal_ip_address_3>:443
 mode tcp
 server app1 tank.shotgunstudio.com:443

Start the service

sudo service haproxy start

If for any reason you need to re-encrypt the traffic, you can switch the proxy mode from tcp to http and use self-signed certificates in order to decrypt older TLS traffic and re-encrypt it. Please note that the CA used to generate the self-signed certificates would need to be added to each client system in order to trust the new certificates. Also, be aware that traffic encryption/re-encryption can introduce security risks at your studio, so proceed with extreme caution.