Interactive evaluation and granular updates

Manipulate and Animate Functionality

WLJS Notebook partially inherits a well-known killer feature of Wolfram Mathematica - Manipulate, Animate, and others, allowing interactive evaluation of arbitrary expressions:

Manipulate[
  Plot[Sin[a x + b], {x, 0, 6}], 
  {{a, 2, "Frequency"}, 1, 4, 1}, 
  {{b, 0, "Phase"}, 0, 10, 1}, 
  ContinuousAction->True
]

(*VB[*)(FrontEndRef["ce2264d6-e74f-462b-9e17-08fcbeb7437b"])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRCeEJBwK8rPK3HNS3GtSE0uLUlMykkNVgEKJ6caGZmZpJjpppqbpOmamBkl6VqmGprrGlikJSelJpmbGJsnAQCG2BXe"*)(*]VB*)

Text strings or any other symbolic expressions can also be manipulated:

Manipulate[Column[{
  Style[StringTemplate["`` m/s"][v], Blue],
  Table["🚗", {i, Floor[v/25]}] // Row 
}], {v,10,100}, ContinuousAction->True]

(*VB[*)(FrontEndRef["654d9ea0-69e9-4448-abea-c729bba1ed7f"])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRCeEJBwK8rPK3HNS3GtSE0uLUlMykkNVgEKm5mapFimJhromlmmWuqamJhY6CYmpSbqJpsbWSYlJRqmppinAQCGahZA"*)(*]VB*)

Under the hood, it uses a diff algorithm to interpret changes between old and new expressions evaluated using corresponding parameters. Then, if possible, the expression is transformed into a more optimized one that uses granular updates instead of full re-evaluation.

Animate works in a similar way, but scans the parameter automatically:

Animate[
  ParametricPlot[ReIm @ Exp[-I (\[Phi] + \[Gamma] I \[Phi])], {\[Phi],0,5Pi}, 
    PlotLabel->StringTemplate["\[Gamma] = ``"][\[Gamma]]
  ]
, {\[Gamma],0,0.5}]

(*VB[*)(FrontEndRef["ac784f29-dc89-4ee5-86a8-582b311c6466"])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRCeEJBwK8rPK3HNS3GtSE0uLUlMykkNVgEKJyabW5ikGVnqpiRbWOqapKaa6lqYJVromloYJRkbGiabmZiZAQCHwxVc"*)(*]VB*)

For symbolic data it works too:

Animate[{x, (*SpB[*)Power[x(*|*),(*|*)2](*]SpB*)} // NumberForm, {x,0,10}]

Animate[Row[{Sin[x], "==", Series[Sin[x], {x,0,n}], Invisible[1/2]}], {n, 1, 10, 1}, RefreshRate->3]

If you need to periodically check a symbol or re-evaluate an expression, use Refresh:

Refresh[Now, 1]

It can work for images and many other types as well utilizing the same diff algorithm of Manipulate:

img = ExampleData[{"TestImage", "Lena"}];
Refresh[(img = EdgeDetect[img] 0.05 + img; img), 0.1]

More specialized functions

We recommend using ManipulatePlot and ManipulateParametricPlot for simple interactive curve plotting, since they are more optimized and do not require the diff algorithm to run:

ManipulatePlot[
  (*TB[*)Sum[(*|*)(*FB[*)((Sin[2π(2j - 1) x])(*,*)/(*,*)(2j-1))(*]FB*)(*|*), {(*|*)j(*|*),(*|*)1.0(*|*),(*|*)n(*|*)}](*|*)(*1:eJxTTMoPSmNiYGAoZgMSwaW5TvkVmYwgPguQCCkqTQUAeAcHBQ==*)(*]TB*)
, {x, -1,1}, {{n,4}, 1,7, 1}]

(*VB[*)(FrontEndRef["4580b8e8-5239-41bc-b46e-541b291fce69"])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRCeEJBwK8rPK3HNS3GtSE0uLUlMykkNVgEKm5haGCRZpFromhoZW+qaGCYl6yaZmKXqmgKZRpaGacmpZpYAeLYVTg=="*)(*]VB*)

AnimatePlot and AnimateParametricPlot use a different strategy and precalculate all curves before animating:

AnimateParametricPlot[
    With[{t = If[x > lim,lim,x]},
      N[Exp[-(*FB[*)((1)(*,*)/(*,*)(3))(*]FB*) Floor[(*FB[*)((x)(*,*)/(*,*)(4 \[Pi]))(*]FB*)]] ReIm[(*SpB[*)Power[I(*|*),(*|*)-t](*]SpB*) + 3 ((*SpB[*)Power[I(*|*),(*|*)t/3](*]SpB*))]]
    ]
    , {x, 0.1`, 16 \[Pi]}, {lim, 0.1`, 16 \[Pi]}
]

(*VB[*)(FrontEndRef["7d570729-f4d3-47d7-947e-3022f973c962"])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRCeEJBwK8rPK3HNS3GtSE0uLUlMykkNVgEKm6eYmhuYG1nqppmkGOuamKeY61qamKfqGhsYGaVZmhsnW5oZAQB2dBTQ"*)(*]VB*)

Granular Updates

The way how you update things like points and lines on your plots, images, or bind those elements to the changes of data differs from one used in Pluto.jl, Marimo and Wolfram Mathematica. If the last one uses 2-ways binding, we rely on 1-way. The key changes were made for the sake of performance and simplicity.

Architecture

WLJS mostly uses retained mode for managing objects on screen, which means Line or Point has a dedicated method to update itself when the dependencies change. Not all primitives have an update method implemented—please check our symbols reference section to confirm. The dependencies are resolved and the changes are propagated by the frontend (browser), not by the evaluation kernel. To create such a dependency, you need to "offload" an expression to be executed on the frontend. For example:

l = 1;

Graphics[{Cyan,
  (*BB[*)(Rectangle)(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVFDGDwwb5oDhi8sS+qB4P39gBJMR34"*)(*]BB*)[{-1, -1}, {(*BB[*)(l)(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVFDGDwwb7o7x8QeGNf9PEDCDywBwB4XSLR"*)(*]BB*) // Offload, 1}]
}]

(*VB[*)(Graphics[{RGBColor[0, 1, 1], Rectangle[{-1, -1}, {Offload[l], 1}]}, ImageSize -> {200, 50}])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KWlMIB4HkHAvSizIyEwuhoiA5H0yi0vSmGHyQe5Ozvk5+UWZQDZDJiOMgCjnBClITS5JzEvPSUU1IfM/EIAJNJPB7mAHEv5paTn5iSnFIIEcJENBKoNKc1LBpnvmJqanBmdWoZt+AuQQIyABAKYtMQs="*)(*]VB*)

Here we used Offload to tell Wolfram Kernel not to evaluate symbol l and keep it for evaluating on the frontend. In this example above the binding happens between Rectangle and l. Nothing stops us from offloading a derived expression from the same symbol:

Graphics[{Pink,
  (*BB[*)(Disk)(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVFDGDwwb5oDhi8sS+qB4P39gBJMR34"*)(*]BB*)[{(*BB[*)(l)(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVFDGDwwb7o7x8QeGNf9PEDCDywBwB4XSLR"*)(*]BB*), (*BB[*)(l)(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVFDGDwwb7o7x8QeGNf9PEDCDywBwB4XSLR"*)(*]BB*)}//Offload, (*SpB[*)Power[(*BB[*)(l)(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVFDGDwwb7o7x8QeGNf9PEDCDywBwB4XSLR"*)(*]BB*)(*|*),(*|*)2](*]SpB*) // Offload]
}, PlotRange->{{0,1}, {0,1}}]

(*VB[*)(Graphics[{RGBColor[1, 0.5, 0.5], Disk[Offload[{l, l}], Offload[l^2]]}, PlotRange -> {{0, 1}, {0, 1}}, ImageSize -> 200])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KWnMIB4HkHAvSizIyEwuTmOCyftkFpcg5IPcnZzzc/KLMkH6ixjA4IE9nIHQ5pJZnA2xhB1I+Kel5eQnpqCaWgySzoGQOJSyAomA/PLUIoiqTJAgwpCg0pzUYk6Qkpz8kqDEvPRUNGej8DJBTgS7m7A43GTP3MT01ODMqtTME0AeACW2OtE="*)(*]VB*)

Here the radius of Disk depends on l^2, therefore when l changes, it recalculates the derived square of this symbol. Now let's try to update our symbol:

l = 0.33;

Using a slider we can observe the change easier:

EventHandler[InputRange[0,1,.1], Function[val, l=val]]

(*VB[*)(EventObject[<|"Id" -> "f9254465-73cd-4a66-a54d-97a1a96543a4", "Initial" -> 0.5, "View" -> "13864cb4-9741-4d0f-9746-459803ffa629"|>])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRCeEJBwK8rPK3HNS3GtSE0uLUlMykkNVgEKGxpbmJkkJ5noWpqbGOqapBikgVhmuiamlhYGxmlpiWZGlgByOxTK"*)(*]VB*)

Here we used a built-in UI element InputRange with a basic event handler assigned to it, that updates our symbol with a new value. When a new value is assigned to l symbol, Wolfram Kernel pushes a new value using a fast binary protocol to the frontend, that updates all visible instances coupled to l.

More examples

The provided data does not have to be numerical, it can be textual as well. Here is an example with text spray:

Module[{
  text = "Hello World"
},
  Column[{
    EventHandler[InputText[text], (text = #)&],
    Graphics[Table[{
      RandomColor[],
      Rotate[
        Text[text // Offload, RandomReal[{-1, 1}, 2]],
        RandomReal[{0, 3.14}]
      ]
    }, {40}]]
  }]
]

Demonstration:

One can also stream lists and arrays.

sym[[1]] = 4.0; (* nothing happens *)
sym = sym; (* update! *)

This example showcases a basic FABRIK solver written in 2 cells of Wolfram Language:

ClearAll[handler];
SetAttributes[handler, HoldFirst];
handler[chain_] := Function[target,
  Module[{buffer = chain, origin = {-1,0}, prev = chain, lengths = Norm /@ (chain // Reverse // Differences) // Reverse},
    buffer = Table[With[{p = chain[[-i]]},
      If[i === 1,
        prev = target;
        target
      ,
        prev = prev - Normalize[(prev - p)] lengths[[1-i]];
        prev 
      ]
    ]   
    , {i, chain // Length}] // Reverse;

    buffer = Table[With[{p = buffer[[i]]},
      If[i === 1,
        prev = origin;
        origin
      ,
        prev = prev - Normalize[(prev - p)] lengths[[i-1]];
        prev 
      ]
    ]
    , {i, chain // Length}];

    chain = buffer;
   ]
];

chain = Table[Exp[-ϕ]{-Cos[ϕ], Sin[ϕ]}, {ϕ, 0, π - π/7, π/7.0}];
Graphics[{
    Line[chain // Offload], Black, 
    PointSize[0.04], Point[chain // Offload], Red,
    EventHandler[Point[chain // Last], {
      "drag" -> handler[chain]
    }]
  },
  Axes->True, PlotRange->{{-1,0.2}, {0,0.4}}
]

Here is a demonstration:

The beauty of this approach is that we do not need to run the update cycle continuously, but only when a user drags a control point.

3D graphics features in updates are more limited compared to 2D, but you can still do many things with the simple Offload technique. For example, manipulate radii of Tube sections:

tube = Table[5.0, {40}];

Graphics3D[{
  LABColor[0.2,0.4,0.5],
  Directive["Roughness"->0.0, "Ior"->5.0],
  Tube[
    Table[20.0{-Cos[2Pi i/39.0],Cos[2Pi i/39.0],Sin[2Pi i/39.0]}, {i,1.0,Length[tube]}], 
    Offload @ tube
  ]
}]

and run the update cycle:

Do[
tube = Sum[
  Table[.5+5. (*SpB[*)Power[Sin[(*FB[*)((i-j+k)(*,*)/(*,*)(39.0))(*]FB*) Pi i/20.0](*|*),(*|*)2](*]SpB*), {j,Length[tube]}], 
  {k,0,3 40,40}]/2.0;
Pause[0.02];
, {i, 1, 100.0, 0.25}];

Here is how it looks:

Here is another example combining event handling and Offload:

Colors and Opacity

In Wolfram Language color, opacity and other directives are automatically applied to all primitives, even scoped with a List appearing after them, i.e.:

{Red, (*BB[*)(Disk[...])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVF8WDw3r5IAAwe2BfNAYM79gBQ7x14"*)(*]BB*), Blue, {(*BB[*)(Disk[...])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVF0mBw277o108QeGlfdP8eCLyzBwCCPCLr"*)(*]BB*), (*BB[*)(Line[...])(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVF0mBw277o108QeGlfdP8eCLyzBwCCPCLr"*)(*]BB*)}}

Therefore if we use dynamic symbols and offload them to the frontend, there will be 2 types of binding:

opacity = 1.0;
Graphics[{
  Triangle[],
  (*BB[*)(Opacity)(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVFDGDwwb5oDhi8sS+qB4P39gBJMR34"*)(*]BB*)[(*BB[*)(opacity)(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVFDGDwwb7o7x8QeGNf9PEDCDywBwB4XSLR"*)(*]BB*) // Offload], 
  Red, (*BB[*)(Disk)(*,*)(*"1:eJxTTMoPSmNkYGAoZgESHvk5KRAeB5AILqnMSXXKr0hjgskHleakFnMBGU6JydnpRfmleSlpzDDlQe5Ozvk5+UVFd26DgX3Rgvkg8N6+6NJFEHhoDwDL+yT7"*)(*]BB*)[{0,0}, 0.25]
}]

from opacity to Opacity directive, and from Disk to opacity. When we update opacity, the whole chain will be pulled:

opacity = RandomReal[{0,1}];

To update RGBColor or Hue you need to bind them in the same way:

color = {1.0,1.0,1.0};
Graphics[{RGBColor[color//Offload], Disk[]}]

For convenience, you may apply the following pattern:

color = List@@RGBColor[Pink];

color = List@@RGBColor[LABColor[0.6,0.5,0.5]];

Animation

Low-level animation can be done in two ways:

Transitions

This one is reserved exclusively for Graphics or higher-order functions like Plot* or HighlightImage, which produce Graphics. Any update to a primitive is interpolated on the frontend by default. You can control it with TransitionType and TransitionDuration directives, set globally as an option to Graphics or locally using Directive. Here is an example:

pos = {-1,0};

Graphics[{
  Directive["TransitionType"->None],
  Disk[(pos+{0,0.5})//Offload, 0.1],
  Directive["TransitionType"->"Linear"],
  Disk[(pos)//Offload, 0.1],
  Directive["TransitionType"->"CubicInOut"],
  Disk[(pos+{0,-0.5})//Offload, 0.1]
}, 
  PlotRange->{{-1,1}, {-1,1}},
  "TransitionDuration"->2000
]

Then update the positions instantly:

pos = -pos;

This interpolation can also help visually if Wolfram Kernel can't keep up with updates in real-time or your animation looks choppy. Most primitive properties can be interpolated, except color and opacity.

Frames

The naive way of animating things could be something like a loop with Pause:

Do[data = animate[i]; Pause[0.01], {i, 99999}]

This will work. However, it is a suboptimal approach: it blocks the kernel, it is not in sync with the screen refresh (which may cause flickering), and there is no reliable way to embed it in a presentation or simply suspend it.

To solve these issues for any type of animated graphics, use AnimationFrameListener. It does a few things:

Here is an example with many 3D spheres animated:

balls = RandomReal[{-1,1}, {100,3}];
vels  = RandomReal[{-1,1}, {100,3}];

Graphics3D[{
  Table[With[{i = i},
    {
      RGBColor[RandomReal[{0,1}, 3]],
      Sphere[balls[[i]] // Offload, 0.03]
    }
  ], {i, Length[balls]}], 

  EventHandler[AnimationFrameListener[balls // Offload], handler]
}]

Here balls is passed as a dependency to AnimationFrameListener, acting as a trigger that signals the animation is complete and needs to fire an event handled by the handler function for the next frame. Let's define our basic handler function that updates the ball positions:

handler[_] := With[{},
  vels = Table[
    If[Norm[balls[[i]]] < 0.01, -1, 1] vels[[i]] - 0.08 balls[[i]]
  , {i, Length[balls]}];
  
  balls = Table[balls[[i]] + 0.08 vels[[i]], {i, Length[balls]}];
]

To start the animation, you need to evaluate the first cell we described or kick-start it by manually triggering an update:

balls = balls;

To interrupt an animation, you can simply break the cycle:

handler[_] := Null;

Here is another nice and easy example with 2D points:

pts = RandomReal[{-1,1}, {100, 2}];
Graphics[{
  Point[pts // Offload],
  EventHandler[AnimationFrameListener[pts//Offload], Function[Null,
    pts = Map[(# + 0.01 Normalize[{-1,1} #[[{2,1}]]])&, pts];
  ]]
}, TransitionType->None]

In some cases, timers can also be used. If your animation is slow and you do not need to be in sync with the screen, use SetInterval. Here is a previous example rewritten with timers:

TaskRemove[task] // Quiet;

Module[{
  path = {{0,0}}
},
  task = SetInterval[
    path = Append[path, Last[path] + RandomChoice[{
      {0,1}, {0,-1}, {1,0}, {-1,0}
    }]];
  , 100];

  Graphics[{
    Line[path//Offload]
    }, PlotRange->20{{-1,1},{-1,1}},
    "TransitionType"->None
  ]
]

However, now you are responsible for stopping the task.

Combine with Animate

You can combine the power of high-level and low-level animation methods by overriding the update function of Animate. For example:

Module[{pts},
  Animate[Graphics[{
    Circle[{0,0},1],
    Red, Point[pts // Offload]
  }, ImageSize->Small], {t, 0, 2Pi, 0.1}, 
  "UpdateFunction" -> Function[t,
    pts = {Sin[t], Cos[t]};
    False
  ]]
]

Here UpdateFunction returns False, which prevents the default update cycle of Animate and applies only our pts assignment instead. As an advantage, you get UI controls for your custom animation for free. You can improve the previous example by increasing the refresh rate and disabling interpolation:

Module[{pts},
  Animate[Graphics[{
    Circle[{0,0},1],
    Red, Point[pts // Offload]
  }, ImageSize->Small,
    "TransitionType"->None
  ], {t, 0, 2Pi, 0.04}, 
  "UpdateFunction" -> Function[t,
    pts = {Sin[t], Cos[t]};
    False
  ], RefreshRate->60]
]

How to keep and update many objects

In the example above, we tried the ideal cases when one can substitute many data points into a single primitive. In reality, you might need to give them different properties, colors, and so on. For this, we need to address each element by index explicitly and wrap everything with Offload:

pts = RandomReal[{-1,1}, {100, 2}];
Graphics[
	Table[With[{i=i}, {Point[pts[[i]]] // Offload}], {i, 100}]
]

or like this:

pts = RandomReal[{-1,1}, {100, 2}];
Graphics[
	Table[With[{i=i}, {Point[pts[[i]] // Offload]}], {i, 100}]
]

In both cases we keep the expressions pts[[1]], pts[[2]] ... to be evaluated on the frontend.

We don't dynamically create points; we statically create them on Wolfram Kernel and offload the position of each point.

Let's now stylize our points:

pts = RandomReal[{-1,1}, {400, 2}];
col[r_] := With[{n=Clip[Norm[r]/2]}, RGBColor[1-n, n, n]]

Graphics[
	Table[With[{i=i}, {col[pts[[i]]], Point[pts[[i]]] // Offload}], {i, 400}]
]

Do[
  pts = Map[(# + 0.01 Normalize[{-1,1} #[[{2,1}]]])&, pts];
  Pause[1/27.0];
, {50}];

With zero-cost we can generate more points using the same pts using simple math operations

pts = RandomReal[{-1,1}, {400, 2}];
col[r_] := With[{n=Clip[Norm[r]/2]}, RGBColor[1-n, n, n]]

Graphics[
	Table[With[{i=i}, {col[pts[[i]]], Point[1.1 pts[[i]]] // Offload, Point[pts[[i]]] // Offload}], {i, 400}]
]

Multidimensional arrays

The idea is the same; however, you have to be careful with the Part or [[]] operator. Our frontend engine does not fully support all the complex syntax of Wolfram Language. Here is a perfectly valid expression evaluated on Wolfram Kernel:

multipts[[i,j]]

while for the frontend you should do this:

multipts[[i]][[j]] // Offload

Here is an example with a multidimensional array, where the first part stores color and the second one stores the position:

mpts = {Clip[Norm[#]], #} &/@ RandomReal[{-1,1}, {400, 2}];

Graphics[
	Table[With[{i=i}, {
       Hue[mpts[[i]][[1]]], 
       Point[mpts[[i]][[2]]]
    } // Offload], {i, 400}], 
	"TransitionDuration"->30
]

Do[
  mpts = Map[{
    If[#[[1]]>0.95, 0, #[[1]] + 0.025],
    #[[2]] + 0.01 Normalize[{-1,1} #[[2, {2,1}]]]
  }&, mpts];
  Pause[1/30.0];
, {100}];

How to append/remove objects

For basic cases, Line, Point, and Polygon naturally support variable numbers of points or vertices:

points = {{0,0}};
Graphics[{Point[points//Offload]}, PlotRange->{{-1,1},{-1,1}}]

Do[
  points = Append[points, Last[points]+RandomReal[0.1{-1,1},2]];
  Pause[0.03];
, {100}];

Appending

To append new complex objects to an existing visible plot or graphical canvas (i.e., Graphics, Graphics3D), you need to reference it and submit an expression to be evaluated there as if it were present from the beginning.

The particular context of the instance can be referenced with FrontInstanceReference:

ref = FrontInstanceReference[];
Plot[x, {x,0,1}, Prolog->{
  ref
}]

ref will point directly to what is inside Prolog. If any color directives are present, they will be applied automatically to the submitted expressions.

To submit anything for frontend evaluation, use FrontSubmit. Using the second optional argument, we specify where to evaluate the expression. For example, to add a red circle on our existing plot:

FrontSubmit[{Red, Disk[{0.5,0.5}, 0.5]}, ref]

FrontSubmit[ZoomAt[1.2, {0.5,0.5}], ref] 

Here is another example, let's call it Bubbles:

ref = FrontInstanceReference[];
EventHandler[Plot[x, {x,0,1}, Epilog->{
  ref
}], {"mousemove" -> Function[xy,
  FrontSubmit[{
    RandomColor[],
    Disk[xy, {0.7, 1} RandomReal[{0.01, 0.1}]]
  }, ref]
]}]

Removing

To append and remove some objects, you need to create a group first - FrontInstanceGroup

group = FrontInstanceGroup[];

Plot[x, {x,0,1}, Prolog->{
  group[Disk[{0,0}, 1]]
}]

Later to remove it use - FrontInstanceGroupRemove or Delete:

FrontInstanceGroupRemove[group]

How to update raster graphics

Raster graphics is mainly rendered using the output form of the Image expression, which is a resource-efficient container for any image-like data with a large set of built-in methods. Image is immutable, but it also accepts expressions wrapped inside Offload to be dynamically displayed:

testImg = ExampleData[{"TestImage", "Lena"}]//Thumbnail;
imageBuffer = ImageData[testImg];
Image[imageBuffer // Offload, "Real32"]

By default, ImageData returns a float representation of pixel data, therefore we explicitly set "Real32". Let's update it by applying blur to our image:

imageBuffer = ImageData[testImg//Blur];

Here is another example, where we actually generate pixel data directly on Wolfram Kernel (cloud generation) and stream it:

ClearAll[n, k2, spectrum, im, p0, p, buffer];

n = 256;
k2 = Outer[Plus, #, #] &[RotateRight[N@Range[-n, n - 1, 2]/n, n/2]^2];

spectrum = With[{d := RandomReal[NormalDistribution[], {n, n}]},
   (1/n) (d + I d)/(0.000001 + k2)]; 
spectrum[[1, 1]] *= 0;

im[p_] := Clip[Re[InverseFourier[spectrum Exp[I p]]], {0, ∞}]^0.5

p0 = p = Sqrt[k2];

buffer = im[p0 += p];

Image[buffer // Offload, "Real32", Epilog->{
    EventHandler[AnimationFrameListener[buffer // Offload], 
      Function[Null, buffer = im[p0 += 2 p]]
    ]
}]

The performance will not be the best in either of these cases, since floating-point arrays are resource-demanding, but for displaying color, 3 bytes is usually enough. To solve this issue, we recommend using NumericArray typed to "Byte". For example:

testImage = ExampleData[{"TestImage", "Lena"}];
toBuf[image_Image] := NumericArray[ImageData[image, "Byte"],"Byte"];

buffer = testImage // toBuf;
Image[buffer//Offload, "Byte"]

and then we perform Opening with a provided radius

EventHandler[InputRange[0,50,.1], Function[r,
  buffer = Opening[testImage, r] // toBuf;
]]

Here is another example with manual pixel manipulation. First - extract raw pixel data:

img = ImageData[ExampleData[{"TestImage", "Lena"}], "Byte"];

Now we apply a mapping function to it, which stretches and shrinks pixels periodically:

shader = Compile[{{img, _Integer, 3}, {phase, _Real}}, Module[{iter},
  With[{
    ysize = Length[img],
    xsize = Length[img[[1]]]
  },
    Table[
      iter = 1.0;
      Table[With[{
        yr = y,
        xr = If[# < 1, 1, If[# > xsize, xsize, #]] &@ Round[iter]
      },

        iter = iter + (1.0 + 0.7 Sin[6 Pi x / xsize + phase]);

        img[[yr, xr]]
        
      ], {x, xsize}], {y, ysize}]
  ]
]];

Here, we use Compile to speed up the process since only integer arrays are involved. We do not apply any antialiasing filters; it works in the nearest neighbors approximation. Let's check the result using the original Byte encoding:

Image[NumericArray[shader[img, 0], "Byte"], "Byte"] 

Since performance of our shader function may not be great, we can use a straightforward Do loop:

imageFrame = NumericArray[img, "Byte"];
Image[imageFrame // Offload, "Byte"]

Do[
  imageFrame = NumericArray[shader[img, angle], "Byte"];
, {angle, 0, 4Pi, 2Pi/30.0}]

The resulting animation:

Beyond visuals

Many other expressions support live updates.

Progress bar

ProgressIndicator represents a complete progress bar indicator, which can be made live using Offload:

p = 0.0;
ProgressIndicator[p//Offload, {0,1}]

Do[p = i; Pause[0.3];, {i,0,1.0,0.1}];

You can use it together with CellPrint to temporarily show it while some long task is running. Here is a helper function:

reporter[max_:1.0] := Module[{p=0.0, timer = AbsoluteTime[]}, With[{
  cell = CellPrint[ProgressIndicator[p//Offload, {0,max}]]
},

  object["Set", val_] := With[{now = AbsoluteTime[]},
    If[now - timer > 0.1,
      p = val;
      timer = now;
    ];
  ];
  
  object["Finished"] := (
    NotebookDelete[cell];
    ClearAll[object, p];
  );
  
  object
] ]

Now we can use it in some long-running procedure:

Module[{bar = reporter[]},
  With[{result = Table[

    bar["Set", i];
    Pause[0.01];
    i

  , {i, 0, 1.0, 0.01}]},
    bar["Finished"];
    result
  ]
]

Text and editor views

To display changing textual or numerical data, use TextView:

text = "";
TextView[text // Offload]

text = RandomWord[];

Here text can be String, Real, Integer, or a list of values. For dynamically rendering HTML, use HTMLView in the same way.

In cases where you need to display changing Wolfram Expressions, use EditorView. EditorView accepts only strings as data input.

text = "";
EditorView[text // Offload]

To get a nice-looking expression in StandardForm convert it to a string using ToString:

Do[text = ToString[x^n, StandardForm]; Pause[0.08], {n, 1, 10}];

It can be extremely powerful, since you can practically put any expression:

text = ToString[Plot[x, {x,0,1}], StandardForm];

Audio stream

The output form of Audio objects can be used for live raw audio streaming using Offload expression similar to Image:

buffer = {};
Audio[buffer//Offload]

buffer = Table[Exp[-(*FB[*)((x)(*,*)/(*,*)(100.))(*]FB*)] Sin[(*FB[*)((x)(*,*)/(*,*)(1.5))(*]FB*)], {x, 0, 100 \[Pi], 0.1}];

By default, Audio assumes "Real32" (mono) format of input data and SampleRate equals 44100.

Audio objects in streaming mode can be used with EventHandler to track when the buffer is about to be exhausted. For example, for continuous streaming:

buffer = {};
samplingFunction = Function[t, Exp[-Mod[t,3]/1.0]Sin[200.0 t]]; 
time = 0;

EventHandler[Audio[buffer//Offload, "SignedInteger16", SampleRate->8000], {
"More" -> Function[Null,
  With[{c = (1.0/8000.0) (2Pi)},
    With[{sampled =  32760 Table[samplingFunction[c (i + time)], {i, 0, 4 1024 - 1}]},
      buffer = NumericArray[sampled, "SignedInteger16", "ClipAndRound"];
    ];
    
    time += 4 1024;
  ];
]}]