LibAV instances are created by an asynchronous factory function, LibAV.LibAV.
In many cases, it can be called with no option, e.g., libav = await LibAV.LibAV(). LibAV.LibAV takes an optional argument in which loading
options may be provided. The loading options and their default values are:
{
"noworker": false,
"nowasm": false,
"yesthreads": false,
"nothreads": false,
"base": <automatically detected>,
"toImport": <automatically computed>,
"factory": <automatically imported>,
"variant": <specified by libav.js filename>,
"wasmurl": <automatically computed>
}
nowasm forces libav.js to load only asm.js code, not WebAssembly code. By
default, it will determine what the browser supports and choose accordingly, so
this is overridable here mainly for testing purposes.
The other no/yes options affect the execution mode of libav.js. libav.js can run
in one of three modes: "direct" (synchronous), "worker", or "threads".
After creating a libav.js instance, the mode can be found in
libav.libavjsMode. By default, libav.js will use the "worker" mode if
Web Workers are available, and "direct" otherwise. libav.js never uses the
"threads" mode by default, though this may change in the future.
If noworker is set or Web Workers are not available, Web Workers will be
disabled, so libav.js will run in the main thread (i.e., will run in "direct"
mode). This is synchronous, so usually undesirable. Note that if you're loading
libav.js in a worker, it may be reasonable to set noworker, and make
libav.js synchronous with your worker thread.
If yesthreads is set (and nothreads is not set) and threads are supported
(see
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer
), then a threaded version of libav.js will be loaded. This will significantly
improve the performance of some encoders and decoders. However, threads are
disabled by default, as their benefit or otherwise depends on the precise
behavior of your code, and some browsers have a fairly low limit to the number
of worker threads an entire page is allowed to have. Note that separate
instances of libav.js, created by separate calls to LibAV.LibAV, will be in
separate threads as long as workers are used, regardless of the value of
yesthreads, and thus yesthreads is only needed if you need concurrency
within a libav.js instance.
libav.js automatically detects which WebAssembly features are available, so even
if you set yesthreads to true, a version without threads may be loaded. To
know which version will be loaded, call LibAV.target. It will return "asm"
if only asm.js is used, "wasm" for baseline, or "thr" for threads. These
strings correspond to the filenames to be loaded, so you can use them to preload
and cache the large WebAssembly files. LibAV.target takes the same optional
argument as LibAV.LibAV.
The base option can be used in these options in place of LibAV.base, and
will override LibAV.base if set.
The variant option can be set to load a different variant than the one given
by the URL you loaded libav.js itself from, which in turn can be used to load
multiple variants at the same time.
The toImport, factory, and wasmurl options are documented in the
“bundlers” section of the main README, as they are mainly of
interest to bundlers.
The tests used to determine which features are available are also exported, as
LibAV.isWebAssemblySupported and LibAV.isThreadingSupported.
NOTE: libav.js used to have a SIMD build as well. This was dropped because none of the constituent libraries actually support WebAssembly SIMD, so it substantially increased the size and time of builds to no benefit.
The LibAV.LibAV factory returns (a promise resolving to) a libav instance,
which is an object exposing libav and libav.js's API as methods.
Most of libav.js's API is libav's API, and for such functions, you can consult FFmpeg's documentation. Not every function is exposed, of course; see funcs.json for a list of exposed functions or to add new functions.
Functions that use double-pointers are exposed as _js metafunctions that
take and return single pointers.
Most structs are exposed as raw pointers (numbers), and their parts can be
accessed using accessor functions named Struct_member and Struct_member_s.
For instance, to read frame_size from an AVCodecContext, use await AVCodecContext_frame_size(ctx), and to write it, use await AVCodecContext_frame_size_s(ctx, frame_size). There are also libav.js-specific
JavaScript objects for many of them, documented in libav.types.d.ts.
Further examples are available in the samples directory of
https://github.com/ennuicastr/libavjs-webcodecs-polyfill , which uses libav.js
along with WebCodecs (or its own polyfill of WebCodecs), so shows how to marry
these two technologies.
The following additional functionality is provided by libav.js itself, divided here by the libav component it belongs to. Please read ../libav.types.in.d.ts for type declarations.
libav frames and packets can be represented as numbers, pointers to actual
AVFrame or AVPacket instances, but most functions will instead copy out
their data into libav.js Frame or Packet objects.
One important similarity in Frame and Packet objects is the
libavjsTransfer member. If you send a Frame or Packet into libav.js (and
it happens to be running a worker), then any ArrayBuffers given in the
libavjsTransfer array will be transferred. By default, all included
ArrayBuffers are in libavjsTransfer, so all will be transferred. This works in
both directions, so if you get a Frame from libav.js and then send it back
again, you will lose access to its underlying data.
For the rest of the details of these types, see the TypeScript type documentation.
ff_init_muxer(
opts: {
oformat?: number,
format_name?: string,
filename?: string,
device?: boolean,
open?: boolean,
codecpars?: boolean
},
streamCtxs: [number, number, number][]
): Promise<[number, number, number, number[]]>
Initializes a muxer all at once, opening the file and initializing the format
context. You can provide the format as a libav numerical code (oformat) or as
a name (format_name), the filename to write to (which can of course be a
device) (filename), and/or create it as a writer device automatically
(device). You have the option not to open the file (open=false, the
default), in which case you will need to provide your own pb.
For the streams to mux, each stream is in the form [number, number, number],
consisting of the codec context, time_base_num, and time_base_den,
respectively. If opts.codecpars is set, use a codec parameters (codecpar), not
a codec context.
Returns [output context (oc), format, writer context (pb), stream contexts].
Usually called as [oc, fmt, pb] = await ff_init_muxer(...).
To write, you must first use libav.avformat_write_header, and after writing
all packets, you must use libav.av_write_trailer. You may write packets with
libav.ff_write_multi (below), or directly using libav APIs.
ff_write_multi(
oc: number, pkt: number, inPackets: (Packet | number)[], interleave?: boolean
): Promise<void>
Write packets to an output context. You need to not just provide the packet(s)
(inPackets), but allocate space for packets in libav format, so there's
somewhere to write them to temporarily (pkt). Packets may be in libav.js
Packet format, or AVPacket pointers. Use av_packet_alloc (and eventually,
av_packet_free) for that, or get it from one of the AVCodec metafunctions.
interleave means that it will use av_interleaved_write_frame, and if
interleave===false, it will use av_write_frame instead. interleave
defaults to true, and this is usually the right option, but if your input is
already interleaved, you should set this to false.
ff_free_muxer(oc: number, pb: number): Promise<void>
Frees the context generated by ff_init_muxer.
ff_init_demuxer_file(
filename: string, fmt?: string
): Promise<[number, Stream[]]>
Initializes a demuxer from the given filename, which can be a reader device. Optionally takes the expected format as a string.
Returns [format context (fmt_ctx), streams]. Streams are of the Stream type.
Note that there's no equivalent of ff_free_muxer, because
ff_init_demuxer_file only initializes one libav object. Free fmt_ctx with
avformat_close_input_js.
ff_read_frame_multi(
fmt_ctx: number, pkt: number, opts?: {
limit?: number, // OUTPUT limit, in bytes
unify?: boolean, // If true, unify the packets into a single stream (called 0), so that the output is in the same order as the input
copyoutPacket?: string // Version of ff_copyout_packet to use
}
): Promise<[number, Record<number, Packet[]>]>
Read some packets from a format context. Like ff_write_multi, you must provide
pkt.
By default, this will read as much as it can, which is typically the entire file. If using a device in this default mode, you will need to feed it data, and won't get anything back until it's done.
To limit how much data is sent, use the limit option. limit limits how much
data ff_read_frame_multi outputs to one packet more than the number of bytes
you request.
For reading from a reader device, see also ff_reader_dev_waiting and
ff_reader_dev_send.
Returns [result, packets]. The result is the result code from the underlying
read; 0 is success, but you can also expect -libav.EAGAIN (if you hit a
limit) or libav.AVERROR_EOF (at the end of the file).
The returned packets are in a record indexed by the stream index. Those indices
are the indices of the Stream objects in the Stream[] array given by
ff_init_demuxer_file. Alternatively, you can use unify, in which case all
packets will be in input order in a single array, packets[0].
There are multiple versions of ff_copyout_packet, only one of which is
actually used to copy out packets. See the documentation of ff_copyout_packet
below for how to use the copyoutPacket option.
ff_copyout_packet(pkt: number): Promise<Packet>
Variants: ff_copyout_packet_ptr
Copy a packet from internal libav memory (pkt) as a libav.js object.
The ff_copyout_packet_ptr function is also available, and copies the packet
into a separate AVPacket pointer, instead of actually copying out any data.
This is a good compromise if you're building pipelines, e.g. reading then
decoding, to avoid copying data back and forth when that data is just going back
into libav.js. Be careful, though! ff_read_frame_multi reads from every
stream, and if you're only using data from one of them, copied packets using
ff_copyout_packet_ptr will leak memory! Use ff_copyout_packet_ptr carefully.
Metafunctions that use ff_copyout_packet internally, namely
ff_read_frame_multi, have a configuration option, copyoutPacket, to specify
which version of ff_copyout_packet to use. It is a string option, accepting
the following values: "default", "ptr".
ff_copyin_packet(pktPtr: number, packet: Packet | number): Promise<void>
Copy a packet as a libav.js object (packet) into libav memory (pktPtr). Also
works to duplicate a packet that is already an AVPacket pointer as a number.
ff_init_encoder(
name: string, opts?: {
ctx?: AVCodecContextProps, options?: Record<string, string>
}
): Promise<[number, number, number, number, number]>
Allocate and initialize an encoder (whether audio or video). Sometimes, all that
is sufficient is the name, in libav style, e.g. libopus or aac. Common
features like bitrate are in ctx (see the description of AVCodecContextProps
in libav.types.in.d.ts). options is for codec-specific options.
Returns a lot of things, some of which aren't always needed: [codec, codec context (c), frame, packet (pkt), frame size]. Usually called as [, c, frame, pkt, frame_size] = await ff_init_encoder(...).
ff_encode_multi(
ctx: number, frame: number, pkt: number, inFrames: (Frame | number)[],
fin?: boolean
): Promise<Packet[]>
Encode multiple frames into packets. Set fin if these are the last frames;
otherwise the arguments should be obvious. Note that it's fine to set inFrames
to [] to encode no frames, typically to set fin. The frames may be AVFrame
pointers, as numbers.
ff_free_encoder(
c: number, frame: number, pkt: number
): Promise<void>
Free the things allocated by ff_init_encoder.
ff_init_decoder(
name: string | number, config?: {
codecpar?: number | CodecParameters,
time_base?: [number, number]
}
): Promise<[number, number, number, number]>
Initialize a decoder. name can be a string name (e.g. "libopus") or an
internal identifier. Usually, an internal identifier would come from a Stream
from ff_init_demuxer_file, in which case codecpar and time_base would also
come from that Stream.
Returns a lot of things, some of which aren't always needed: [codec, codec context (c), packet (pkt), frame]. Usually called as [, c, pkt, frame] = ff_init_decoder(...).
ff_decode_multi(
ctx: number, pkt: number, frame: number, inPackets: (Packet | number)[],
config?: boolean | {
fin?: boolean,
ignoreErrors?: boolean,
copyoutFrame?: string
}
): Promise<Frame[]>
Decode multiple packets, which may be libav.js Packets or AVPacket pointers.
config can be set to true as fin, which means these are the last packets.
Alternatively, config can be set to an object, and ignoreErrors will attempt
to continue decoding in the case of errors.
There are multiple versions of ff_copyout_frame, only one of which is actually
used to copy out frames. See the documentation of ff_copyout_frame below for
how to use the copyoutFrame option.
Combination of ff_decode_multi and ff_filter_multi. Documented with
ff_filter_multi, below.
ff_free_decoder(
c: number, pkt: number, frame: number
): Promise<void>
Free the things allocated by ff_init_decoder.
ff_copyout_frame(frame: number): Promise<Frame>
Variants: ff_copyout_frame_video, ff_copyout_frame_video_packed,
ff_copyout_frame_video_imagedata, ff_copyout_frame_ptr
Copy a frame out of internal libav memory (frame) as a libav.js object.
ff_copyout_frame supports video frames, but if you know a frame is a video
frame, you can bypass the check by using ff_copyout_frame_video instead.
When video frames are copied out as Frames, the frame's data is copied as a
single Uint8Array, and the layout of the pixel data within that data is given
by a layout array. Each frame in the layout is an element of the array (a
PlaneLayout), and each PlaneLayout consists of an offset and a stride. The
offset is the location of this plane in the data, and the stride is the number
of bytes between lines, which can be fairly arbitrary for memory alignment
reasons.
ff_copyout_frame_video_packed copies out the same as ff_copyout_frame_video,
but with all the data packed (i.e., with the minimum possible stride).
To copy out video frames directly as ImageData objects instead of libav.js
Frames at all, use ff_copyout_frame_video_imagedata. ImageData is only
available in browsers. Further, ff_copyout_frame_video_imagedata does not
convert the image format, so video frames must already be in RGBA (not RGB32!)
format to use it.
The ff_copyout_frame_ptr function is also available, and copies the frame into
a separate AVFrame pointer, instead of actually copying out any data. This is
a good compromise if you're building pipelines, e.g. decoding and then
filtering, to avoid copying data back and forth when that data is just going
back into libav.js.
Metafunctions that use ff_copyout_frame internally, namely ff_decode_multi
and ff_filter_multi, have a configuration option, copyoutFrame, to specify
which version of ff_copyout_frame to use. It is a string option, accepting the
following values: "default", "video", "video_packed", "ImageData", "ptr".
ff_copyin_frame(framePtr: number, frame: Frame | number): Promise<void>
Copy a libav.js Frame object (frame) into libav memory (framePtr). Also
works if frame is another AVFrame pointer, e.g. as created by
ff_copyout_frame_ptr.
ff_init_filter_graph(
filters_descr: string,
input: FilterIOSettings | FilterIOSettings[],
output: FilterIOSettings | FilterIOSettings[]
): Promise<[number, number | number[], number | number[]]>;
Create a filter graph. Most of the context is filters_descr is either a simple
description (as would be passed to -vf or -af in ffmpeg) or a complex
description (as would be passed to -filter_complex).
FilterIOSettings describes things like the sample rate/frame rate and sample
format/pixel format. A filter graph can have multiple inputs and multiple
outputs, in which case input and output can be arrays.
If there is one input, the input pad is named in. Otherwise, the input pads
are named in0, in1, etc. If there is one output, the output pad is named
out. Otherwise, the output pads are named out0, out1, etc.
Returns [filter graph, source context(s), sink context(s)]. If there are
multiple inputs (sources), then source contexts is an array, and if there are
multiple outputs (sinks), then sink contexts is an array.
ff_filter_multi@sync(
srcs: number | number[], buffersink_ctx: number, framePtr: number,
inFrames: (Frame | number)[] | (Frame | number)[][], config?: boolean | {
fin?: boolean,
copyoutFrame?: string
}
): @promise@Frame[]@;
Filter one or more frames. Takes one or more source contexts (srcs), but only
one sink context (buffersink_ctx). If you need to filter with multiple
outputs, you need to use the lower-level libav functions. inFrames should be
an array of frames if there's only one source, or an array of arrays if there
are multiple sources. Frames may be libav.js Frames or AVFrame pointers as
numbers.
Also requires a frame pointer (framePtr) to use for temporary storage, which
can be allocated directly or by an encoding/decoding metafunction.
config can be used to pass configuration options, or pass true as config
as an equivalent of config.fin. Set fin if these are the last input frames.
There are multiple versions of ff_copyout_frame, only one of which is actually
used to copy out frames. See the documentation of ff_copyout_frame above for
how to use the copyoutFrame option.
ff_decode_filter_multi(
ctx: number, buffersrc_ctx: number, buffersink_ctx: number, pkt: number,
frame: number, inPackets: (Packet | number)[],
config?: boolean | {
fin?: boolean,
ignoreErrors?: boolean,
copyoutFrame?: string
}
): Promise<Frame[]>
Combination of ff_decode_multi and ff_filter_multi in a single function.
Useful to avoid the overhead of copying data and waiting for Promises between
decoding and filtering.
await ff_decode_filter_multi(ctx, src, sink, pkt, frame, packets, config) is
equivalent to await ff_filter_multi(src, sink, frame, await ff_decode_multi(ctx, pkt, frame, packets), config). That is, this really just
combines the two calls. However, internally, neither frame copying nor Promises
are used.
The readFile, writeFile, unlink, and mkdev functions are provided
directly from Emscripten's filesystem module. In addition, functions to create
streaming devices are provided. These are further documented in IO.md.
mkreaderdev(name: string, mode?: number): Promise<void>
Make a reader device. This is used to stream data, and acts like a Unix character device. The mode is usually unnecessary.
mkblockreaderdev(name: string, size: number): Promise<void>
Similar to mkreaderdev, but mkreaderdev creates a character device
(streaming device), whereas mkblockreaderdev creates a block device, so it
will have a size and random access.
To intercept read requests from the block reader device(s), you must set
libav.onblockread to a function (name: string, position: number, length: number) => void.
Because onblockread is a callback, it is usually possible to create a block
reader device and its callback and then use the API as if no devices are used.
ff_reader_dev_send(name: string, data: Uint8Array): Promise<void>
Send data to a reader device. There is no limit imposed by libav.js to how much data a reader device can buffer.
ff_block_reader_dev_send(
name: string, position: number, data: Uint8Array
): Promise<void>
Similar to ff_reader_dev_send, but for block reader devices, and has a
position specified for the data.
ff_reader_dev_waiting(name?: string): Promise<boolean>
Returns true if one or more reader or block reader devices are waiting for
data. Give the filename to check if a specific reader device is waiting, or no
name to check if any reader device is waiting.
Typically, this is used along with ff_init_demuxer_file and
ff_read_frame_multi to feed in data. If instead of awaiting those promises,
you store the promises aside, you can loop with while (await libav.ff_reader_dev_waiting()) and send data, then await the file-reading
promise. In this way, you can control the transfer of input data without having
to send entire files or predict how much data might be needed.
mkwriterdev(name: string, mode?: number): Promise<void>
Make a writer device. This is generally used to stream data, but some formats aren't streamable, so this acts a bit more like a Unix block device.
To receive data from the writer device(s), you must set libav.onwrite to a
function (name: string, position: number, buffer: Uint8Array) => void.
mkstreamwriterdev(name: string, mode? number): Promise<void>
Make a stream writer device. Identical to a writer device except that seeking
is not allowed, so libav treats it like a stream. Most formats either don't
care or simply won't work with a stream, but certain formats (like wav and
matroska) will behave differently if the output is a stream than if it's a
block device.
Receive data in the same way as with mkwriterdev.
If a variant is used that employs the cli fragment, then the entire ffmpeg
and ffprobe CLIs are exposed, as well as the various libav interfaces.
ffmpeg(...args: (string | string[])[]): Promise<number>
Runs the ffmpeg CLI tool. args can be each string argument to ffmpeg, or
an array of strings, or any combination thereof. Returns ffmpeg's exit code.
NOTE: ffmpeg 6.0 and later require threads for the ffmpeg CLI. libav.js does support the ffmpeg CLI on unthreaded environments, but to do so, it uses a thread compatibility layer, emfiberthreads. This is just a compatibility layer, and is expected not only to be slower than true multithreading, but to be slower than single-threaded operation.
ffprobe(...args: (string | string[])[]): Promise<number>
Like ffmpeg, but for the ffprobe tool.