From the Trenches: Multiple chunked uploads

For those of you who find yourselves dealing with larger document packets, you may have found yourself hitting a block with the file size capacity for our document uploads. This capacity does differ slightly pending on which platform you’re presently using, but the range tends to be between 25 and 32MB; larger payloads than this will be rejected. So then, what do you do when you find yourself working with a larger packet, say 40MB or 50MB overall?

Our solution for handling larger file sizes like this would be chunked uploads. If you have not already, I would recommend having a look at the previous Trenches post on the topic to familiarize yourself before reading further.

For ease of access, here are the rules for using this system I gave in the previous post:

• Chunked uploads expire after 20 minutes or on use.
• Chunked uploads have an optional checksum parameter to do a final integrity check.
• Chunked uploads can have a total file size accumulation of 1GB at any given time. This limit is for your account, not for individual envelopes.
• The maximum file size for an accumulated chunked upload is 52MB.

Now, it’s important to remember that the larger these envelopes and document packets become, the longer they’re going to take to load and/or interact with our system. Note that the individual pages inside of an envelope must be rendered one at a time, so depending on the recipients’ machine and local internet speeds, having a huge number of documents can lead to slowdowns and syncing issues when dealing with lower bandwidth connections.

Another thing to take into consideration is the way that Docusign tabs operate. When a tab is signed, there is a background telemetry tracker that registers tab values and what has been changed. To make a long story short, the more tabs you use, the more delay between when you click to sign a tab vs. when our system reacts to it.

Last, keep in mind that you can use multiple chunked uploads within the same envelope; however, if the overall size of the envelope documents exceeds 200MB, you will receive an error message, as this is an internal limit.

Interesting, but how does this impact me?

Good question! Let’s take a look at a practical example.

Let’s say that you’re a bank or an investment firm with some pretty hefty-sized document packets. For the major deal you’re about to close, the overall number of these packets is about five; however, all five of the packets are about 30MB apiece in overall file size. You already know from experience that, if you try to use any of these packets via the UI or via a standard createEnvelope call, you’re going to receive an error message, so you decide to use chunked uploads. You also know that you want to avoid lengthy processes within your application and would rather have it going in sequence instead of all at once; that way, if something does go wrong in terms of timeouts, error messages, etc., you can follow the sequence piece-by-piece instead of having to interpret the entire process all at once. You also know that you don’t want the envelope to send until you’re ready; that way, if something does happen, your recipient isn’t notified they have a contract to sign when elements are missing.

From this, you can extrapolate a workflow:

1. Create and commit a chunked upload.
2. Use the first chunked upload to create an envelope with a status of created (draft):

     {
       "documents": [
         {
           "remoteUrl": "$chunkedUploadURI",
           "documentId": "1",
           "name": "Example.txt"
         }
       ],
       "status": "created"
     }

3. Create and commit the next chunk, then add it to the envelope:

     {
       "documents": [
         {
           "remoteUrl": "$chunkedUploadURI",
           "documentId": "$newDocumentId",
           "name": "$additionalDocumentName.pdff"
         }
       ]
     }

4. Repeat step 3 until all five have been added.
5. Add recipients and anchor tabs:

     "signers": [
       {
         "email": "example@email.com",
         "name": "Example User 1",
         "recipientId": "1",
         "routingOrder": "1",
         "tabs": {
           "signHereTabs": [
             {
               "recipientId": "1",
               "tabLabel": "Example Anchor String",
               "anchorString": "\\String\\",
               "anchorUnits": "pixels"
             }
           ]
         }
       }
     ],

6. Verify that the envelope was created as intended. You can accomplish this in one of two ways:
   • With a senderViewToken, which would allow one of your representatives to inspect the envelope and confirm everything is in order by sending it off. (Note that Sender Views are meant for non-admin users who are trusted members of your company; see Secure Coding Practices: Lock Down Your Sender View! For additional information.)
   • With a getEnvelope call including URL parameters for include=recipients,tabs. This will return the envelope status and the recipient information, as well as the populated tab details.
7. Shift the envelope status from created to sent, which will finalize the envelope, make it live, and send out any associated email notifications. This is the step where your potential recipients can begin interacting with the envelope that you’ve created: {"status":"sent"}

Now that you have a workflow outlined, the next thing you need to do is make sure that the workflow is practical and attempt to identify potential issues prior to beginning QA testing. To this purpose, I’ve created a test app to implement this workflow and inspect the results.

One item that did become very apparent during this initial test was the amount of time it took for the anchor strings and recipients to actually apply. The overall envelope size ended up being around 175MB, with roughly 1,000 pages and 3,000 tabs attached. The application of the recipient and associated anchor tabs took approximately eight minutes in demo, which should be significantly lower in production.

Based on this, there are a few things to keep in mind:

1. Do not exceed 200MB; this is the system limit and you will receive an error message.
2. Do not use common anchor strings; use specific strings for specific tab types. You can place these strings and hide them by matching the font color to the background of the document.
3. Based on the processing time, you may want to consider breaking up calls to add individual recipients or individual tab types to lower the processing time. This will help avoid the potential for timeouts.
4. When using chunked uploads, the required number of API calls to add a document to an envelope becomes four instead of one. Docusign limits the number of hourly API transactions an account can make by default to 1,000 calls per hour. Using chunked uploads is considered a legitimate workflow; however, this particular envelope took about 20 API calls to create it. What this means is that, with the default capacity, I would be limited to sending roughly 50 envelopes per hour. If you’re planning on adopting this as a standard workflow, you’re going to need to reach out to the Developer Support team to have them take a look at the workflow you’ve come up with, and more importantly, to get that limit increased so that you can send off a significantly larger number of envelopes before hitting your limits.

Using chunked uploads can be fairly tricky; however, this is presently the only way to upload documents between 25 and 50MB in overall size. If you’re having trouble getting the workflow up and running, feel free to reach out to Customer Support referencing this blog post. I or another member of my team would be happy to help you get this new workflow up and running.

Additional resources

• eSignature REST API Reference: ChunkedUploads resource
• Docusign Developer Center
• @DocuSignAPI on Twitter
• Docusign for Developers on LinkedIn
• Docusign for Developers on YouTube
• Docusign Developer Newsletter