
Most websites rely heavily on server-side calls for gathering information & media processing, often resulting in users frequently waiting on loading screens. Since the Possible Studio also supports real-time collaboration, building UX faces many of the same challenges as adopting real-time multiplayer into a game:
* Real-time collaboration means that multiple clients need to send, receive and reconcile updates on the same media objects \- similar to how gamers need to see other gamer movements without delay  
* Media generation means server-side updates can take a while, which takes longer to sequence \- similar to how gamers can perform many actions in parallel that need to be synchronized accurately

We leverage two key principles to solve this problem:
* **Optimistic UI** is a design pattern that updates the interface immediately with predicted results of a server operation, enhancing perceived responsiveness. The UI is then updated again with the actual server response when received, ensuring data consistency. This approach significantly improves the user experience by reducing apparent latency while maintaining data accuracy.  
* **Eventual Consistency** is a distributed system model where updates propagate asynchronously across nodes. It guarantees that all clients will converge to the same state given enough time without new updates, prioritizing availability over immediate consistency. This approach allows for temporary inconsistencies but ensures all nodes eventually reflect the latest data.

Lets take a look at a working example for a key flow in the Possible Studio:
1. User writes a *shot*  
2. This *shot* text is used along with other story information to generate images  
3. These images are processed (cropped/resized/copied) based on certain story settings  
4. If the user chooses a *shot* action, the processed images are then further reprocessed to generate various video snippets and merge them together

![](/blog/media/optimistic-ui-1.jpg)

Relying purely on our backend servers for media processing would result in multiple collaborators waiting many seconds to view generated images and modify *shot* actions.

Instead, we leverage optimistic UI & eventual consistency:
1. The web client that the user is on, directly handles the image generation process (this process is a 3rd party call which cannot be avoided so it must be low latency)  
2. Once this web client has the image it sends it to the server for processing \- however, it first stores all the image processing information using Yjs+Firestore  
3. Since the Yjs+Firestore sync is extremely low latency, this update is received by all the other web clients a lot faster than the server-side processed image  
4. Each web client then renders its own simulation of the “processed” image using HTML canvas & media elements, until the server-side processed image is available  
5. Once available, all web clients swap out to use the server image

![](/blog/media/optimistic-ui-2.jpg)

What happens if the user chooses a *shot* action while the server is still processing the initial image? Leveraging the same principles:
1. The server-side video generation process behaves like another client \- it grabs the image processing information from Yjs+Firestore  
2. It then adds this information to its own processing parameters and these changes are all run as one update so latency is unaffected for processed vs unprocessed images

A future improvement to bring into the app is to have each web client also handle most video generations using HTML canvas & media elements. This will let us further extend the window of eventual consistency \- multiple clients need not be consistent for 10-20 seconds since the user will see an accurate simulation of the video that the server-side processing is currently generating.
