OCI: Please, Try Again Later

Recently, I ran into an interesting problem: the system attempts to read the bucket before the new object is available. Fortunately, the OCI SDK offers retry policies and rules, allowing you to take control over retry strategies.

I did some code refactoring (a fancy name for a total rewrite of the component), and as a result, my class went completely asynchronous. The new and improved functionality led to the new roadblock. With full asynchronicity, the system tries to read an object before an OCI bucket completes an upload. Unfortunately, I can't control the code usage; all I can do is implement one of the retry strategies.

The good news is that the OCI SDK provides this functionality at the client or at the operation level. TypeScript/JavaScript and Python libraries provide similar mechanisms for defining complex behavioral patterns to handle timeouts, errors, and delays. In my scenario, all I need is a small lag between the moment when objects are uploaded, the method returns a new resource URI, and the time when the OCI bucket makes them available for access. Since I don't want to retry all operations, I will add retry configurations only to "exists" and "read" operations.

The code below shows how to use the retry configuration. For the sake of clarity, I stripped out all surrounding code, the constructor, authentication, and service functions.

async exists(fileName, targetDir) {
  const targetName = 
    buildPath(this.getTargetDir(this.pathPrefix), targetDir, fileName);
  const headOp = {
      objectName: stripLeadingSlash(targetName),
      bucketName: this.bucket,
      namespaceName: this.namespace,
      retryConfiguration: {
        retryCondition:  common.DefaultRetryCondition,
        terminationStrategy: new   
          common.MaxAttemptsTerminationStrategy(this.retryAttempts)
       } 
  }
  return this.build().then(storage => {
    storage.headObject(headOp)
       .then((result) => { 
                return true;
       })
       .catch((err) => {    
         if (err.statusCode === 404) {
            return false;
         } else {
            throw err;
         }
       });
  });
}

I altered the default retry strategy to terminate after a defined number of retry attempts. By default, the client will try eight times before giving up (according to the Python SDK). It seems a bit high to me, so I'll set max retries to three. The termination strategy may be one of the preconfigured options, or you may define a custom one using custom retry configuration and various strategies.

Tests from my local machine demonstrate that the retry configuration works, and on the second retry attempt, my client reads the object from the OCI bucket.

An extra configuration option for the OCI client, or an additional configuration parameter, allows you to save time and effort on implementing custom error handling. SDK offers a comprehensive set of predefined strategies and conditions. I'll give you a hint: the Python documentation is way better than the [Type|Java]Script one, so you may want to read the Python SDK documentation to grasp the overall idea and apply it to the JavaScript code.