Integrating Data Insights into System Models

Making System Models Data-Driven

Modern system models shouldn’t exist in a vacuum—they thrive on data. By integrating real-world information, models become far more data-driven, reflecting actual system behaviour and results.

In systems engineering, this is crucial. Consider a scenario where you run physical tests on a prototype and gather performance metrics. Rather than manually inputting those metrics into your model, you can link your model directly to the test data. This enables closed-loop design iterations: you refine the model based on test results, simulate new conditions, and feed insights back into design updates.

In this fourth installment of the Simulating Systems with OPCloud series, we’ll explore how to seamlessly pull data from a database into your system model, integrating valuable insights with each simulation.

If you missed the earlier parts, you can catch up on the series through our newsletter here.

Understanding Relational Databases

A relational database stores information in a structured format, arranging data neatly into tables composed of rows and columns—much like spreadsheets. What sets relational databases apart is their capability to interconnect these tables through unique identifiers known as keys. These keys enable efficient querying across multiple tables, making it simpler to combine and extract meaningful insights from complex datasets.

SQL, or Structured Query Language, serves as the standard method for interacting with these databases, providing intuitive commands that let you effortlessly retrieve, insert, update, or delete records. By structuring data in interconnected tables, relational databases empower users to efficiently handle complex queries, enhancing data clarity and reliability.

Step-by-Step Guide to Integrating SQL Databases with OPCloud

In this section, we’ll walk through connecting an SQL database to your OPCloud model. While specific setup details can vary, the core principles remain the same. The key idea is to make your OPCloud model communicate with a database through a WebSocket interface, enabling real-time queries and updates during simulation.

For a hands-on example, we’ve prepared a Docker image with a SQL database pre-loaded with test data. If you’d like to experiment in a cloud environment without local setup, feel free to message me for a link to a Deepnote notebook that hosts this setup for testing.

To illustrate the integration, imagine we have a dataset of experiment results stored in a database table. Each record includes: an experiment_id, temperature, voltage, current, and efficiency. Our goal is to retrieve these results for a given experiment and use them inside an OPCloud model.

Step 1: Serve Your Database Via WebSocket

WebSocket is a robust communication protocol designed to enable continuous, real-time data exchange between servers and clients. In this setup, the WebSocket serves as an intermediary, securely facilitating instant communication between OPCloud (the client) and your database (the server).

There are multiple ways to establish the connection. Here, we’re using FastAPI, a lightweight Python framework, to manage the WebSocket communication. This service listens for incoming connections from OPCloud, ensures they are authenticated, and then handles the communication by forwarding queries to the relevant databases.

Once the query is processed, FastAPI quickly sends the results back to OPCloud. This setup enables integrated, real-time data retrieval during simulations, ensuring everything stays in sync and your model always operates with the most current data as it runs.

Step 2: Create OPCloud Model

With the database service ready, the next step is to build your OPCloud model that will interact with it.

In this example, we have a process called “Data Retrieving,” which queries the database and passes the results over to the “Data” object in the model. You can define parameters like “Measurement” and “Condition” as dynamic inputs, allowing the query to adjust depending on the state of the model at any given time. This makes the whole process more flexible and responsive as your simulation evolves.

The “Data Retrieving” process is a computational process configured to execute SQL queries dynamically. For example, the process might run a query like:

SELECT mea FROM experiment_results WHERE con

Here, “mea” and “con” represent the real-time values of the “Measurement” and “Condition” objects. OPCloud automatically inserts these values into the SQL query at runtime, so the data retrieval stays in sync with the changing parameters of your model. This approach makes sure you’re always working with the most relevant data, tailored to the current state of your simulation.

Step 3: Configure WebSocket Connection

After preparing your model, the final step is to set up OPCloud’s database connection. Enter the WebSocket URL, database details, and required credentials in OPCloud’s settings.

Once enabled, OPCloud establishes a WebSocket connection with your database service using these credentials. When executing the model, OPCloud sends the SQL query via the WebSocket. The database service processes this query, retrieves the data from the SQL database, and promptly sends the results back.

OPCloud then updates the “Data” object in your model with the new data. This data can then be immediately used to drive decisions, perform calculations, or validate outcomes dynamically in other parts of the model.

Final Thoughts

Integrating databases with system models significantly boosts the realism and utility of simulations. By bringing your model and data together, OPCloud ensures that your analyses are always grounded in actual, up-to-date data. This setup opens the door to a variety of advanced applications, from live test benches that automatically log results into databases to maintaining digital thread continuity by connecting simulations directly to organisational data sources, making the entire process more cohesive and meaningful.

This data-driven connectivity bridges the gap between theoretical models and real-world conditions. Models become living representations, continuously learning and improving. Moreover, the communication is bidirectional—your model can not only retrieve data but also update the central database, forming a powerful feedback loop. Such integration accelerates learning cycles, facilitates agile development, and ultimately leads to better-engineered systems that continuously adapt and improve.

Interested in learning more about simulation and computation in OPM? Check out our microcourse on OptimiSE Academy!

As we conclude Part 4 of the series, think about the data sources in your own projects. What databases or live data streams could you hook into your system models to make them smarter and more reflective of the real world? What new opportunities do you see for improving your models through data integration? Feel free to share your thoughts below!