FAQ
This page contains the most common questions and "gotchas" asked on our Discord server, in our community calls, or within our community.
There are two major areas we'd like to explore: common errors and helpful hints.
Common errors
Reading errors
If your application doesn't run, and you don't see any error messages on the page, you should check your browser's console.
When reading an error message, the easy way to find out what's going on, most of the time, is to read the last line of the error.
Traceback (most recent call last):
File "/lib/python311.zip/_pyodide/_base.py", line 501, in eval_code
.run(globals, locals)
^^^^^^^^^^^^^^^^^^^^
File "/lib/python311.zip/_pyodide/_base.py", line 339, in run
coroutine = eval(self.code, globals, locals)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<exec>", line 1, in <module>
NameError: name 'failure' is not defined
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'failure' isn't defined
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
In both examples, the code created a
NameError
because the object with the name failure
did not exist. Everything above the
error message is potentially useful technical detail.
With this context in mind, these are the most common errors users of PyScript encounter.
SharedArrayBuffer
This is the first and most common error users may encounter with PyScript:
Failure
Your application doesn't run and in your browser's console you see this message:
When
This happens when you're unable to access objects in the main thread (window
and document
) from code running in a web worker.
This error happens because the server delivering your PyScript application is
incorrectly configured or a service-worker
attribute has not been used in
your script
element.
Specifically, one of the following three problem situations applies to your code:
- Because of the way your web server is configured, the browser limits the use
of a technology called "Atomics" (you don't need to know how it works, just
that it may be limited by the browser). If there is a
worker
attribute in yourscript
element, and your Python code uses thewindow
ordocument
objects (that actually exist on the main thread), then the browser limitation on Atomics will cause the failure, unless you reconfigure your server. - There is a
<script type="py-editor">
(that must always use a worker behind the scenes) and no fallback has been provided via aservice-worker
attribute on that element. - There is an explicit
PyWorker
orMPWorker
instance bootstrapping somewhere in your code and noservice_worker
fallback has been provided.
All these cases have been documented with code examples and possible solutions in our section on web workers.
Why
The only way for document.getElementById('some-id').value
to work in a
worker is to use these two JavaScript primitives:
- SharedArrayBuffer, to allow multiple threads to read and / or write into a chunk of shared memory.
- Atomics,
to both
wait(sab, index)
(sab
is aSharedArrayBuffer
) andnotify(sab, index)
to unlock the awaiting thread.
While a worker waits for an operation on main to happen, it is not using the CPU. It idles until the referenced index of the shared buffer changes, effectively never blocking the main thread while still pausing its own execution until the buffer's index is changed.
As overwhelming or complicated as this might sound, these two fundamental primitives make main ↔ worker interoperability an absolute wonder in term of developer experience. Therefore, we encourage folks to prefer using workers over running Python in the main thread. This is especially so when using Pyodide related projects, because of its heavier bootstrap or computation requirements. Using workers ensures the main thread (and thus, the user interface) remains unblocked.
Unfortunately, we can patch, polyfill, or workaround, these primitives but we cannot change their intrinsic nature and limitations defined by web standards. However, there are various solutions for working around such limitations. Please read our web workers section to learn more.
Borrowed proxy
This is another common error that happens with listeners, timers or in any other situation where a Python callback is lazily invoked from JavaScript:
Failure
Your application doesn't run and in your browser's console you see this message:
When
This error happens when using Pyodide as the interpreter on the main thread, and when a bare Python callable/function has been passed into JavaScript as a callback handler:
import js
# will throw the error
js.setTimeout(lambda msg: print(msg), 1000, "FAIL")
The garbage collector immediately cleans up the Python function once it is passed into the JavaScript context. Clearly, for the Python function to work as a callback at some time in the future, it should NOT be garbage collected and hence the error message.
Info
This error does not happen if the code is executed in a worker and the JavaScript reference comes from the main thread:
from pyscript import window
window.setTimeout(lambda x: print(x), 1000, "OK")
Proxy objects (i.e. how Python objects appear to JavaScript, and vice versa) cannot be communicated between a worker and the main thread.
Behind the scenes, PyScript ensures references are maintained between workers and the main thread. It means Python functions in a worker are actually represented by JavaScript proxy objects in the main thread.
As a result, such worker based Python functions are therefore not bare Python functions, but already wrapped in a managed JavaScript proxy, thus avoiding the borrowed proxy problem.
If you encounter this problem you have two possible solutions:
- Manually wrap such functions with a call to
pyscript.ffi.create_proxy
. - Set the
experimental_create_proxy = "auto"
flag in your application's settings. This flag intercepts Python objects passed into a JavaScript callback and ensures an automatic and sensible memory management operation via the JavaScript garbage collector.
Note
The FinalizationRegistry is the browser feature used to make this so.
By default, it is not observable and it is not possible to predict when it
will free, and hence destroy, retained Python proxy objects. As a result,
memory consumption might be slightly higher than when manually using
create_proxy
. However, the JavaScript engine is responsible for memory
consumption, and will cause the finalization registry to free all retained
proxies, should memory consumption become too high.
Why
PyScript's interpreters (Pyodide and MicroPython) both have their own garbage collector for automatic memory management. When references to Python objects are passed to JavaScript via the FFI, the Python interpreters cannot guarantee such references will ever be freed by JavaScript's own garbage collector. They may even lose control over the reference since there's no infallible way to know when such objects won't be needed by JavaScript.
One solution is to expect users to explicitly create and destroy such proxy objects themselves. But this manual memory management makes automatic memory management pointless while raising the possibility of dead references (where the user explicitly destroys a Python object that's still alive in the JavaScript context). Put simply, this is a difficult situation.
Pyodide provides
ffi.wrappers
to help with many of the common cases, and PyScript, through the
experimental_create_proxy = "auto"
configuration option, automates memory
management via the FinalizationRegistry
described above.
Python packages
Sometimes Python packages, specified via the
packages
configuration setting
don't work with PyScript's Python interpreter.
Failure
You are using Pyodide.
Your application doesn't run and in your browser's console you see this message:
Failure
You are using MicroPython.
Your application doesn't run and in your browser's console you see this message:
When
This is a complicated problem, but the summary is:
- Check you have used the correct name for the package you want to use. This is a remarkably common mistake to make: let's just check. :-)
- In Pyodide, the error indicates that the package you are trying to install has some part of it written in C, C++ or Rust. These languages are compiled, and the package has not yet been compiled for web assembly. Our friends in the Pyodide project and the Python packaging authority are working together to ensure web assembly is a default target for compilation. Until such time, we suggest you follow Pyodide's own guidance to overcome this situation.
- In MicroPython, the package you want to use has not been ported into the
micropython-lib
package repository. If you want to use a pure Python package with MicroPython, use the files configuration option to manually copy the package onto the file system, or use a URL to reference the package.
For hints and tips about packaging related aspects of PyScript read the packaging pointers section of this FAQ.
Why
Put simply, Pyodide and MicroPython are different Python interpreters, and both are running in a web assembly environment. Packages built for Pyodide may not work for MicroPython, and vice versa. Furthermore, if a package contains compiled code, it may not yet have been natively compiled for web assembly.
If the package you want to use is written in a version of Python that both Pyodide and MicroPython support (there are subtle differences between the interpreters), then you should be able to use the package so long as you are able to get it into the Python path via configuration (see above).
Currently, MicroPython cannot expose modules that require native compilation,
but PyScript is working with the MicroPython team to provide different builds
of MicroPython that include commonly requested packages (e.g. MicroPython's
version of numpy
or sqlite
).
Warning
Depending on the complexity of the project, it may be hard to seamlessly make a 1:1 port from a Pyodide code base to MicroPython.
MicroPython has comprehensive documentation to explain the differences between itself and "regular" CPython (i.e. the version of Python Pyodide provides).
JavaScript modules
When using JavaScript modules with PyScript you may encounter the following errors:
Failure
Uncaught SyntaxError: The requested module './library.js' does not provide an export named 'default'
Failure
Uncaught SyntaxError: The requested module './library.js' does not provide an export named 'util'
When
These errors happen because the JavaScript module you are trying to use is not written as a standards-compliant JavaScript module.
Happily, to solve this issue various content delivery networks (CDNs) provide a way to automatically deliver standard ESM (aka: ECMAScript Modules). The one we recommend is esm.run.
<mpy-config>
[js_modules.main]
"https://esm.run/d3" = "d3"
</mpy-config>
<script type="mpy">
from pyscript.js_modules import d3
</script>
Alternatively, ensure any JavaScript code you reference uses export ...
or
ask for an .mjs
version of the code. All the various options and technical
considerations surrounding the use of JavaScript modules in PyScript are
covered in our user guide.
Why
Even though the standard for JavaScript modules has existed since 2015, many old and new libraries still produce files that are incompatible with such modern and idiomatic standards.
This isn't so much a technical problem, as a human problem as folks learn to use the new standard and migrate old code away from previous and now obsolete standards.
While such legacy code exists, be aware that JavaScript code may require special care.
Possible deadlock
Users may encounter an error message similar to the following:
When
This error happens when your code on a worker and in the main thread are in a deadlock. Put simply, neither fragment of code can proceed without waiting for the other.
Why
Let's assume a worker script contains the following Python code:
from pyscript import sync
sync.worker_task = lambda: print('🔥 this is fine 🔥')
# deadlock 💀🔒
sync.main_task()
On the main thread, let's instead assume this code:
<script type="mpy">
from pyscript import PyWorker
def async main_task():
# deadlock 💀🔒
await pw.sync.worker_task()
pw = PyWorker("./worker.py", {"type": "pyodide"})
pw.sync.main_task = main_task
</script>
When the worker bootstraps and calls sync.main_task()
on the main thread, it
blocks until the result of this call is returned. Hence it cannot respond to
anything at all. However, in the code on the main thread, the
sync.worker_task()
in the worker is called, but the worker is blocked! Now
the code on both the main thread and worker are mutually blocked and waiting
on each other. We are in a classic
deadlock situation.
The moral of the story? Don't create such circular deadlocks!
How?
The mutually blocking calls cause the deadlock, so simply don't block.
For example, on the main thread, let's instead assume this code:
<script type="mpy">
from pyscript import window, PyWorker
async def main_task():
# do not await the worker,
# just schedule it for later (as resolved)
window.Promise.resolve(pw.sync.worker_task())
pw = PyWorker("./worker.py", {"type": "pyodide"})
pw.sync.main_task = main_task
</script>
By scheduling the call to the worker (rather than awaiting it), it's possible for the main thread to call functions defined in the worker in a non-blocking manner, thus allowing the worker to also work in an unblocked manner and react to such calls. We have resolved the mutual deadlock.
Helpful hints
This section contains common hacks or hints to make using PyScript easier.
Note
We have an absolutely lovely PyScript contributor called Jeff Glass who maintains an exceptional blog full of PyScript recipes with even more use cases, hints, tips and solutions. Jeff also has a wonderful YouTube channel full of very engaging PyScript related content.
If you cannot find what you are looking for here, please check Jeff's blog as it's likely he's probably covered something close to the situation in which you find yourself.
Of course, if ever you meet Jeff in person, please buy him a beer and remember to say a big "thank you". 🍻
PyScript latest
PyScript follows the CalVer convention for version numbering.
Put simply, it means each version is numbered according to when, in the
calendar, it was released. For instance, version 2024.4.2
was the second
release in the month of April in the year 2024 (not the release on the 2nd
of April but the second release in April).
It used to be possible to reference PyScript via a version called latest
,
which would guarantee you always got the latest release.
However, at the end of 2023, we decided to stop supporting latest
as a
way to reference PyScript. We did this for two broad reasons:
- In the autumn of 2023, we release a completely updated version of PyScript
with some breaking changes. Folks who wrote for the old version, yet still
referenced
latest
, found their applications broke. We want to avoid this at all costs. - Our release cadence is more regular, with around two or three releases a month. Having transitioned to the new version of PyScript, we aim to avoid breaking changes. However, we are refining and adding features as we adapt to our users' invaluable feedback.
Therefore,
pinning your app's version of PyScript to a specific release
(rather than latest
) ensures you get exactly the version of PyScript you
used when writing your code.
However, as we continue to develop PyScript it is possible to get our latest
development version of PyScript via npm
and we could (should there be enough
interest) deliver our work-in-progress via a CDN's "canary" or "development"
channel. We do not guarantee the stability of such versions of PyScript,
so never use them in production, and our documentation may not reflect the
development version.
If you require the development version of PyScript, these are the URLs to use:
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@pyscript/core/dist/core.css">
<script type="module" src="https://cdn.jsdelivr.net/npm/@pyscript/core/dist/core.js"></script>
Warning
Do not use shorter urls or other CDNs.
PyScript needs both the correct headers to use workers and to find its own assets at runtime. Other CDN links might result into a broken experience.
Workers via JavaScript
Sometimes you want to start a Pyodide or MicroPython web worker from JavaScript.
Here's how:
<script type="module">
// use sourceMap for @pyscript/core or change to a CDN
import {
PyWorker, // Pyodide Worker
MPWorker // MicroPython Worker
} from '@pyscript/core';
const worker = await MPWorker(
// Python code to execute
'./micro.py',
// optional details or config with flags
{ config: { sync_main_only: true } }
// ^ just as example ^
);
// "heavy computation"
await worker.sync.doStuff();
// kill the worker when/if needed
worker.terminate();
</script>
from pyscript import sync
def do_stuff():
print("heavy computation")
# Note: this reference is awaited in the JavaScript code.
sync.doStuff = do_stuff
JavaScript Class.new()
When using Python to instantiate a class defined in JavaScript, one needs to
use the class's new()
method, rather than just using Class()
(as in
Python).
Why?
The reason is technical, related to JavaScript's history and its relatively poor introspection capabilities:
- In JavaScript,
typeof function () {}
andtypeof class {}
produce the same outcome:function
. This makes it very hard to disambiguate the intent of the caller as both are valid, JavaScript used to usefunction
(rather thanclass
) to instantiate objects, and the class you're using may not use the modern,class
based, idiom. - In the FFI, the JavaScript proxy has traps to intercept the use of the
apply
andconstruct
methods used during instantiation. However, because of the previous point, it's not possible to be sure thatapply
is meant toconstruct
an instance or call a function. - Unlike Python, just invoking a
Class()
in JavaScript (without thenew
operator) throws an error. - Using
new Class()
is invalid syntax in Python. So there is still a need to somehow disambiguate the intent to call a function or instantiate a class. - Making use of the capitalized-name-for-classes convention is brittle because when JavaScript code is minified the class name can sometimes change.
- This leaves our convention of
Class.new()
to explicitly signal the intent to instantiate a JavaScript class. While not ideal it is clear and unambiguous.
PyScript events
PyScript uses hooks during the lifecycle of the application to facilitate the creation of plugins.
Beside hooks, PyScript also dispatches events at specific moments in the lifecycle of the app, so users can react to changes in state:
m/py:ready
Both the mpy:ready
and py:ready
events are dispatched for every PyScript
related element found on the page. This includes <script type="py">
,
<py-script>
or any MicroPython/mpy
counterpart.
The m/py:ready
events dispatch immediately before the code is executed,
but after the interpreter is bootstrapped.
<script>
addEventListener("py:ready", () => {
// show running for an instance
const status = document.getElementById("status");
status.textContent = 'running';
});
</script>
<!-- show bootstrapping right away -->
<div id="status">bootstrapping</div>
<script type="py" worker>
from pyscript import document
# show done after running
status = document.getElementById("status")
status.textContent = "done"
</script>
A classic use case for this event is to recreate the "starting up"
spinner that used to be displayed when PyScript bootstrapped. Just show the
spinner first, then close it once py:ready
is triggered!
Warning
If using Pyodide on the main thread, the UI will block until Pyodide has finished bootstrapping. The "starting up" spinner won't work unless Pyodide is started on a worker instead.
m/py:done
The mpy:done
and py:done
events dispatch after the either the synchronous
or asynchronous code has finished execution.
<script>
addEventListener("py:ready", () => {
// show running for an instance
const status = document.getElementById("status");
status.textContent = 'running';
});
addEventListener("py:done", () => {
// show done after logging "Hello 👋"
const status = document.getElementById("status");
status.textContent = 'done';
});
</script>
<!-- show bootstrapping right away -->
<div id="status">bootstrapping</div>
<script type="py" worker>
print("Hello 👋")
</script>
Warning
If async
code contains an infinite loop or some orchestration that keeps
it running forever, then these events may never trigger because the code
never really finishes.
py:all-done
The py:all-done
event dispatches when all code is finished executing.
This event is special because it depends upon all the MicroPython and Pyodide scripts found on the page, no matter the interpreter.
In this example, MicroPython waves before Pyodide before the "everything is
done"
message is written to the browser's console.
<script>
addEventListener("py:all-done", () => {
console.log("everything is done");
});
</script>
<script type="mpy" worker>
print("MicroPython 👋")
</script>
<script type="py" worker>
print("Pyodide 👋")
</script>
m/py:progress
The py:progress
or mpy:progress
event triggers on the main thread during
interpreter bootstrap (no matter if your code is running on main or in a
worker).
The received event.detail
is a string that indicates operations between
Loading {what}
and Loaded {what}
. So, the first event would be, for
example, Loading Pyodide
and the last one per each bootstrap would be
Loaded Pyodide
.
In between all operations are event.detail
s, such as:
Loading files
andLoaded files
, when[files]
is found in the optional configLoading fetch
andLoaded fetch
, when[fetch]
is found in the optional configLoading JS modules
andLoaded JS modules
, when[js_modules.main]
or[js_modules.worker]
is found in the optional config- finally, all optional packages handled via micropip or mip will also
trigger various
Loading ...
andLoaded ...
events so that users can see what is going on while PyScript is bootstrapping
An example of this listener applied to a dialog can be found in here.
Packaging pointers
Applications need third party packages and PyScript can be configured to automatically install packages for you. Yet packaging can be a complicated beast, so here are some hints for a painless packaging experience with PyScript.
There are essentially five ways in which a third party package can become available in PyScript.
- The module is already part of either the Pyodide or MicroPython
distribution. For instance, Pyodide includes numpy, pandas, scipy,
matplotlib and scikit-learn as pre-built packages you need only activate
via the
packages
setting in PyScript. There are plans for MicroPython to offer different builds for PyScript, some to include MicroPython's version of numpy or the API for sqlite. - Host a standard Python package somewhere (such as PyScript.com or in a GitHub repository) so it can be fetched as a package via a URL at runtime.
- Reference hosted Python source files, to be included on the file
system, via the
files
setting. - Create a folder containing the package's files and sub folders, and create
a hosted
.zip
or.tgz
/.tar.gz
/.whl
archive to be decompressed into the file system (again, via thefiles
setting). - Provide your own
.whl
package and reference it via a URL in thepackages = [...]
list.
Host a package
Just put the package you need somewhere it can be served (like
PyScript.com) and reference the URL in the
packages
setting. So long as the
server at which you are hosting the package
allows CORS
(fetching files from other domains) everything should just work.
It is even possible to install such packages at runtime, as this example using
MicroPython's mip
tool
demonstrates (the equivalent can be achieved with Pyodide
via micropip
).
# Install default version from micropython-lib
mip.install("keyword")
# Install from raw URL
mip.install("https://raw.githubusercontent.com/micropython/micropython-lib/master/python-stdlib/bisect/bisect.py")
# Install from GitHub shortcut
mip.install("github:jeffersglass/some-project/foo.py")
Provide your own file
One can use the files
setting to copy
packages onto the Python path:
<mpy-config>
[files]
"./modules/bisect.py" = "./bisect.py"
</mpy-config>
<script type="mpy">
import bisect
</script>
Code archive (zip
/tgz
/whl
)
Compress all the code you want into an archive (using either either zip
or
tgz
/tar.gz
). Host the resulting archive and use the
files
setting to decompress it onto
the Python interpreter's file system.
Consider the following file structure:
Host it somewhere, and decompress it into the home directory of the Python interpreter:
<mpy-config>
[files]
"./my_module.zip" = "./*"
</mpy-config>
<script type="mpy">
from my_module import util
from my_module.sub import sub_util
</script>
Please note, the target folder must end with a star (*
), and will contain
everything in the archive. For example, "./*"
refers to the home folder for
the interpreter.
File System
Python expects a file system. In PyScript each interpreter provides its own in-memory virtual file system. This is not the same as the filesystem on the user's device, but is simply a block of memory in the browser.
Warning
The file system is not persistent nor shareable (yet).
Every time a user loads or stores files, it is done in ephemeral memory associated with the current browser session. Beyond the life of the session, nothing is shared, nothing is stored, nothing persists!
Read/Write
The easiest way to add content to the virtual file system is by using native Python file operations:
with open("./test.txt", "w") as dest:
dest.write("hello vFS")
dest.close()
# Read and print the written content.
with open("./test.txt", "r") as f:
content = f.read()
print(content)
Combined with our pyscript.fetch
utility, it's also possible to store more
complex data from the web.
# Assume async execution.
from pyscript import fetch, window
href = window.location.href
with open("./page.html", "wb") as dest:
dest.write(await fetch(href).bytearray())
# Read and print the current HTML page.
with open("./page.html", "r") as source:
print(source.read())
Upload
It's possible to upload a file onto the virtual file system from the browser
(<input type="file">
), and using the DOM API.
The following fragment is just one way to achieve this. It's very simple and builds on the file system examples already seen.
<!-- Creates a file upload element on the web page. -->
<input type="file">
<!-- Python code to handle file uploads via the HTML input element. -->
<script type="mpy">
from pyscript import document, fetch, window
async def on_change(event):
# For each file the user has selected to upload...
for file in input.files:
# create a temporary URL,
tmp = window.URL.createObjectURL(file)
# fetch and save its content somewhere,
with open(f"./{file.name}", "wb") as dest:
dest.write(await fetch(tmp).bytearray())
# then revoke the tmp URL.
window.URL.revokeObjectURL(tmp)
# Grab a reference to the file upload input element and add
# the on_change handler (defined above) to process the files.
input = document.querySelector("input[type=file]")
input.onchange = on_change
</script>
Download
It is also possible to create a temporary link through which you can download files present on the interpreter's virtual file system.
from pyscript import document, ffi, window
import os
def download_file(path, mime_type):
name = os.path.basename(path)
with open(path, "rb") as source:
data = source.read()
# Populate the buffer.
buffer = window.Uint8Array.new(len(data))
for pos, b in enumerate(data):
buffer[pos] = b
details = ffi.to_js({"type": mime_type})
# This is JS specific
file = window.File.new([buffer], name, details)
tmp = window.URL.createObjectURL(file)
dest = document.createElement("a")
dest.setAttribute("download", name)
dest.setAttribute("href", tmp)
dest.click()
# here a timeout to window.URL.revokeObjectURL(tmp)
# should keep the memory clear for the session
create_proxy
The create_proxy
function is described in great detail
on the FFI page, but it's also useful to explain when
create_proxy
is needed and the subtle differences between Pyodide and
MicroPython.
Background
To call a Python function from JavaScript, the native Python function needs to be wrapped in a JavaScript object that JavaScript can use. This JavaScript object converts and normalises arguments passed into the function before handing off to the native Python function. It also reverses this process with any results from the Python function, and so converts and normalises values before returning the result to JavaScript.
The JavaScript primitive used for this purpose is the Proxy. It enables "traps", such as apply, so the extra work required to call the Python function can happen.
Once the apply(target, self, args)
trap is invoked:
- JavaScript must find the correct Python interpreter to evaluate the code.
- In JavaScript, the
self
argument forapply
is probably ignored for most common cases. - All the
args
must be resolved and converted into their Python primitive representations or associated Python objects.
Ultimately, the targets referenced in the apply
must exist in the Python
context so they are ready when the JavaScript apply
method calls into the
Python context.
Here's the important caveat: locally scoped Python functions, or functions created at run time cannot be retained forever.
import js
js.addEventListener(
"custom:event",
lambda e: print(e.type)
)
In this example, the anonymous lambda
function has no reference in the Python
context. It's just delegated to the JavaScript runtime via addEventListener
,
and then Python immediately garbage collects it. However, as previously
mentioned, such a Python object must exist for when the custom:event
is
dispatched.
Furthermore, there is no way to define how long the lambda
should be kept
alive in the Python environment, nor any way to discover if the custom:event
callback will ever dispatch (so the lambda
is forever pending). PyScript, the
browser and the Python interpreters can only work within a finite amount of
memory, so memory management and the "aliveness" of objects is important.
Therefore, create_proxy
is provided to delegate responsibility for the
lifecycle of an object to the author of the code. In other words, wrapping the
lambda
in a call to create_proxy
would ensure the Python interpreter
retains a reference to the anonymous function for future use.
Info
This probably feels strange! An implementation detail of how the Python
and JavaScript worlds interact with each other is bleeding into your code
via create_proxy
. Surely, if we always just need to create a proxy, a
more elegant solution would be to do this automatically?
As you'll see, this is a complicated situation with inevitable tradeoffs,
but ultimately, through the
experimental_create_proxy = "auto"
flag,
you probably never need to use create_proxy
. This section of
our docs gives you the context you need to make an informed decision.
However, this isn't the end of the story.
When a Python callback is attached to a specific JavaScript instance (rather than passed as argument into an event listener), it is easy for the Python interpreter to know when the function could be freed from the memory.
from pyscript import document
# logs "click" if nothing else stopped propagation
document.onclick = lambda e: print(e.type)
"Wait, wat? This doesn't make sense at all!?!?", is a valid question/response to this situation.
In this case there's
no need to use create_proxy
because the JavaScript reference to which the
function is attached isn't going away and the interpreter can use the
FinalizationRegistry
to destroy the lambda
(or decrease its reference count) when the underlying
JavaScript reference to which it is attached is itself destroyed.
In Pyodide
The create_proxy
utility was created
(among others)
to smooth out and circumvent the afore mentioned memory issues when using
Python callables with JavaScript event handlers.
Using it requires special care. The coder must invoke the destroy()
method
when the Python callback is no longer needed. It means coders must track the
callback's lifecycle. But this is not always possible:
- If the callback is passed into opaque third party libraries, the reference is "lost in a limbo" where who-knows-when the reference should be freed.
- If the callback is passed to listeners, timers or promises it's hard to predict when the callback is no longer needed.
Luckily the Promise
use case is automatically handled by Pyodide, but we're
still left with the other cases:
from pyscript import ffi, window
# The create_proxy is needed when a Python
# function isn't attached to an object reference
# (but is, rather, an argument passed into
# the JavaScript context).
# This is needed so a proxy is created for
# future use, even if `print` won't ever need
# to be freed from the Python runtime.
window.setTimeout(
ffi.create_proxy(print),
100,
"print"
)
# This is needed because the lambda is
# immediately garbage collected.
window.setTimeout(
ffi.create_proxy(
lambda x: print(x)
),
100,
"lambda"
)
def print_type(event):
print(event.type)
# This is needed even if `print_type`
# is not a scoped / local function.
window.addEventListener(
"some:event",
ffi.create_proxy(print_type),
# despite this intent, the proxy
# will be trapped forever if not destroyed
ffi.to_js({"once": True})
)
# This does NOT need create_function as it is
# attached to an object reference, hence observed to free.
window.Object().no_create_function = lambda: print("ok")
To simplify this complicated situation PyScript has an
experimental_create_proxy = "auto"
flag. When set, PyScript intercepts
JavaScript callback invocations, such as those in the example code above, and
automatically proxies and destroys any references that are garbage collected
in the JavaScript environment.
When this flag is set to auto
in your configuration, you should never need
to use create_proxy
with Pyodide.
Note
When it comes code running on a web worker, due to the way browser work, no Proxy can survive a round trip to the main thread and back.
In this scenario PyScript works differently and references callbacks
via a unique id, rather than by their identity on the worker. When running
on a web worker, PyScript automatically frees proxy object references, so
you never need to use create_proxy
when running code on a web worker.
In MicroPython
The proxy situation is definitely simpler in MicroPython. It just creates
proxies automatically (so there is no need for a manual create_proxy
step).
This is because MicroPython doesn't (yet) have a destroy()
method for
proxies, rendering the use case of create_proxy
redundant.
Accordingly, the use of create_proxy
in MicroPython is only needed for
code portability purposes between Pyodide and MicroPython. When using
create_proxy
in MicroPython, it's just a pass-through function and doesn't
actually do anything.
All the examples that require create_proxy
in Pyodide, don't need it in
MicroPython:
from pyscript import window
# This just works.
window.setTimeout(print, 100, "print")
# This also just works.
window.setTimeout(lambda x: print(x), 100, "lambda")
def print_type(event):
print(event.type)
# This just works too.
window.addEventListener(
"some:event",
print_type,
ffi.to_js({"once": True})
)
# And so does this.
window.Object().no_create_function = lambda: print("ok")
to_js
Use of the pyodide.ffi.to_js
function is described
in the ffi page.
But it's also useful to cover the when and why to_js
is needed, if at
all.
Background
Despite their apparent similarity, Python dictionaries and JavaScript object literals are very different primitives:
ref = {"some": "thing"}
# Keys don't need quoting, but only when initialising a dict...
ref = dict(some="thing")
const ref = {"some": "thing"};
// Keys don't need quoting, so this is as equally valid...
const ref = {some: "thing"};
In both worlds, accessing ref["some"]
would produce the same result: the
string "thing"
.
However, in JavaScript ref.some
(i.e. a dotted reference to the key) would
also work to return the string "thing"
(this is not the case in Python),
while in Python ref.get("some")
achieves the same result (and this is not the
case in JavaScript).
Perhaps because of this, Pyodide chose to convert Python dictionaries to
JavaScript
Map
objects that share a
.get
method
with Python.
Unfortunately, in idiomatic JavaScript and for the vast majority of APIs,
an object literal (rather than a Map
) is used to represent key/value pairs.
Feedback from our users indicates the dissonance of using a Map
rather than
the expected object literal to represent a Python dict
is the source of a
huge amount of frustration. Sadly, the APIs for Map
and object literals
are sufficiently different that one cannot be a drop in replacement for
another.
Pyodide have provided a way to override the default Map
based behaviour, but
this results some rather esoteric code:
import js
from pyodide.ffi import to_js
js.callback(
to_js(
{"async": False},
# Transform the default Map into an object literal.
dict_converter=js.Object.fromEntries
)
)
Info
Thanks to a
recent change in Pyodide,
such Map
instances are
duck-typed to behave like
object literals. Conversion may not be needed anymore, and to_js
may just
work without the need of the dict_converter
. Please check.
MicroPython's version of to_js
takes the opposite approach (for
many of the reasons stated above) and converts Python dictionaries to object
literals instead of Map
objects.
As a result, the PyScript pyscript.ffi.to_js
ALWAYS returns a JavaScript
object literal by default when converting a Python dictionary no matter if
you're using Pyodide or MicroPython as your interpreter. Furthermore, when
using MicroPython, because things are closer to idiomatic JavaScript behaviour,
you may not even need to use to_js
unless you want to ensure
cross-interpreter compatibility.
Caveat
Warning
When using pyscript.to_js
, the result is detached from the original
Python dictionary.
Any change to the JavaScript object will not be reflected in the original Python object. For the vast majority of use cases, this is a desirable trade-off. But it's important to note this detachment.
If you're simply passing data around, pyscript.ffi.to_js
will fulfil your
requirements in a simple and idiomatic manner.