Running out of memory on a machine learning workbench
When executing a script in a machine learning workbench (either manually or via scheduling), you may encounter an issue where the script fails due to your machine learning workbench running out of memory. When this error occurs, it means that the RAM resources of your machine learning workbench have been exceeded.
Use the solutions below to free up additional RAM resources to run your script. If the issue is not resolved, you may need to open a support ticket to acquire more resources.
Potential solutions
If there are multiple scripts scheduled to run using the same machine learning workbench, it causes them to share the RAM resources. Ensure each script has completed before any subsequent scripts are executed. It is recommended that you add a buffer of time between script executions to ensure they do not overlap, such as adding a 15 minute buffer between when one script completes and the next script starts.
To ensure all variables in memory are cleared before executing your notebook, add a new cell at the beginning and enter %reset -fs
For example: %reset magic command documentation
Open the machine learning workbench where this notebook is scheduled to run.
Navigate to the Kernels tab.
Shut down any unneeded kernels and terminals to maximize the amount of available RAM.
Go to the Machine Learning Workbench app on the Machine Learning page.
Click on the three dots menu and select Shutdown.
Run the script again to determine if the issue persists.
If multiple scripts are being run from the same workbench, they share the available resources. If the out-of-memory error occurs, consider moving some of the scripts to other workbenches. Running only one script in a specific workbench ensures that no other scripts will impact the resources available for processing.
As the size of your data increases, your script will need more RAM to process and execute successfully. It is recommended that you review the following third party resources to optimize your script's memory utilization:
With the dedicated resource tier, increasing the RAM resources for the workbench is possible. Use the Configurations tab in the Machine Learning section of your Celonis team to increase the amount of RAM. If you are interested in upgrading to a dedicated resource tier, open a support case to discuss your options.
For more help
If the issue persists after completing the steps above, or to inquire about upgrading to a dedicated resource tier, open a support request in Celopeers. Ensure the following information is included in your support request:
Workbench URL
Screenshots of the issue
If using PyCelonis, identify and include the version
To identify your current version, open the workbench terminal and execute:
pip show PyCelonis.
A copy of the notebook that is failing to execute
Open the notebook.
Manually run the notebook to reproduce the issue and click Save.
Go to File and select the "Download" option.
If an HTML file is downloaded, this is your confirmation page. Open the HTML file and follow the instructions to download the notebook.