1.5. Render Engines

class raysect.core.workflow.RenderEngine[source]

Provides a common rendering workflow interface.

This is a base class, its functionality must be implemented fully by the deriving class.

This class provides a rendering workflow that abstracts away the underlying system performing the work. It is intended that render engines may be built that provide rendering on single cores, multi-cores (SMP) and clusters.

The basic workflow is as follows. The render task is split into small, self-contained chunks of work - ‘tasks’. These tasks are passed to the render engine which distributes the work to the available computing resources. These discrete computing resources are know as “workers”. Workers process one task at a time and return their result to the render engine. When results are received the render engine assembles them into the final result.

This workflow is implemented by supplying a set of tasks and two methods to the render engines’ run() method which processes those tasks. The functions supplied to the run() method may be given additional args and kwargs.

A worker calls render for each task object received. render has the following signature:

def render(task, *render_args, **render_kwargs)

where args and kwargs are additional arguments supplied by the user.

Similarly, the worker calls update() for the results generated by a call to render(). Update() has the following signature:

def update(results, *update_args, **update_kwargs)

where args and kwargs are additional arguments supplied by the user.

The render() function must return an object representing the results, this must be a picklable python object.

The execution order of tasks is not guaranteed to be in order. If the order is critical, an identifier should be passed as part of the task definition and returned in the result. This will permit the order to be reconstructed.

run(tasks, render, update, render_args=(), render_kwargs={}, update_args=(), update_kwargs={})[source]

Starts the render engine executing the requested tasks.

Parameters

  • tasks (list) – List of user defined tuples that describe the task to execute.

  • render (object) – Callable python object that executes the tasks.

  • update (object) – Callable python object that is called following a render task and must be used to update the internal state of the object requesting work.

  • render_args (tuple) – Additional arguments to pass to user defined render function.

  • render_kwargs (tuple) – Additional keyword arguments to pass to user defined render function.

  • update_args (tuple) – Additional arguments to pass to user defined update function.

  • update_kwargs (tuple) – Additional keyword arguments to pass to user defined update function.

worker_count()[source]

Returns the number of workers in use by this engine.

class raysect.core.workflow.SerialEngine[source]

Bases: raysect.core.workflow.RenderEngine

Render engine for running on a single CPU processor.

This engine is useful for debugging.

>>> from raysect.core import SerialEngine
>>> from raysect.optical.observer import PinholeCamera
>>>
>>> camera = PinholeCamera((512, 512))
>>> camera.render_engine = SerialEngine()
class raysect.core.workflow.MulticoreEngine(processes=None, tasks_per_job=None, start_method='fork')[source]

Bases: raysect.core.workflow.RenderEngine

A render engine for distributing work across multiple CPU cores.

The number of processes spawned by this render engine is controlled via the processes attribute. This can also be set at object initialisation.

If the processes attribute is set to None (the default), the render engine will automatically set the number of processes to be equal to the number of CPU cores detected on the machine.

If a render is being performed where the time to compute an individual task is comparable to the latency of the inter process communication (IPC), the render may run significantly slower than expected due to waiting for the IPC to complete. To reduce the impact of the IPC overhead, multiple tasks are grouped together into jobs, requiring only one IPC wait for multiple tasks.

By default the number of tasks per job is adjusted automatically. The tasks_per_job attribute can be used to override this automatic adjustment. To reenable the automated adjustment, set the tasks_per_job attribute to None.

Parameters

  • processes – The number of worker processes, or None to use all available cores (default).

  • tasks_per_job – The number of tasks to group into a single job, or None if this should be determined automatically (default).

  • start_method – The method used to start child processes: ‘fork’ (default), ‘spawn’ or ‘forkserver’.

>>> from raysect.core import MulticoreEngine
>>> from raysect.optical.observer import PinholeCamera
>>>
>>> camera = PinholeCamera((512, 512))
>>>
>>> # allowing the camera to use all available CPU cores.
>>> camera.render_engine = MulticoreEngine()
>>>
>>> # or forcing the render engine to use a specific number of CPU processes
>>> camera.render_engine = MulticoreEngine(processes=8)