Skip to Content
PlatformUIMessenger Protocol

UIMessenger Protocol

Fractal’s UIMessenger protocol is designed to allow embedded UI components to securely exchange data with the host application.

Initialization Handshake

UIMessenger prefers to send iframe-parent communication through a dedicated MessageChannel rather than directly through window when possible. The first step of the protocol is then to create a shared MessageChannel.

First, the embedded component pings the parent:

window.parent.postMessage({ type: "UI-lifecycle-iframe-init-handshake" }, '*');

When the parent receives the message, it opens a dedicated MessageChannel and sends an attached MessagePort to the iframe along with any data to render.

const channel = new MessageChannel();
//...
iframe.contentWindow!.postMessage({ type: IframeHandshakeMessageTypes.HOST_REPLY_HANDSHAKE, renderData: args.renderData }, '*', [channel.port2]);

The MessageChannel provides two ports - one for the iframe and one for the parent. Both sides wrap their port in a MessagingClient

Internal MessagingClient

The UIMessenger protocol runs on simple RPC primitives under the hood. The MessagingClient implements these primitives and does all of its communication through the dedicated MessageChannel. The MessagingClient allows the following operations:

// Create MessagingClient from a MessagePort
const mc = new MessagingClient({ port: channel.port1 });
 
// Send an asynchronous request, expecting a response back.
const response = await mc.request({ type: "some-request-type", payload: {} })
 
// "fire and forget" - don't expect a response
mc.emit({ type: "some-event-type", payload: {} })
 
// Handle inbound events
mc.on("some-message-type", (payload: any) => {})

The UIMessenger uses the following message envelope for passing data

export interface ReplySuccessMessageEnvelope {
    kind: 'success';
    id: string;
}
export type ReplySuccessMessage<TData extends Record<string, unknown> = Record<string, unknown>> = ReplySuccessMessageEnvelope & TData;
export interface ReplyErrorMessage {
    kind: 'error';
    id: string;
    error: string;
}
export interface EventMessageEnvelope {
    kind: 'event';
    id: string;
}
export type EventMessage<TData extends Record<string, unknown> = Record<string, unknown>> = EventMessageEnvelope & TData;
export interface RequestMessageEnvelope {
    kind: 'request';
    id: string;
}
export type RequestMessage<TData extends Record<string, unknown> = Record<string, unknown>> = RequestMessageEnvelope & TData;
export type MessageEnvelope<TData extends Record<string, unknown> = Record<string, unknown>> = RequestMessage<TData> | ReplySuccessMessage<TData> | ReplyErrorMessage | EventMessage<TData>;

The data passed within the message envelope is of type UIActionMessage | AgentActionMessage.

export type UIActionResizeMessage = {
    type: 'resize';
    payload: {
        width: number;
        height: number;
    };
};
export type UIActionToolCallMessage = {
    type: 'tool';
    payload: {
        toolName: string;
        params: Record<string, unknown>;
    };
};
export type UIActionPromptMessage = {
    type: 'prompt';
    payload: {
        prompt: string;
    };
};
export type UIActionLinkMessage = {
    type: 'link';
    payload: {
        url: string;
    };
};
export type UIActionIntentMessage = {
    type: 'intent';
    payload: {
        intent: string;
        params: Record<string, unknown>;
    };
};
export type UIActionNotificationMessage = {
    type: 'notify';
    payload: {
        message: string;
    };
};
export type AgentActionListTools = {
    type: "listTools";
    payload: {};
};
export type AgentActionCallTool = {
    type: "callTool";
    payload: {
        [x: string]: unknown;
        name: string;
        _meta?: {
            [x: string]: unknown;
            progressToken?: string | number | undefined;
        } | undefined;
        arguments?: {
            [x: string]: unknown;
        } | undefined;
    };
};
export type AgentActionQueryDOM = {
    type: "queryDom";
    payload: {
        selector?: string;
    };
};
export type AgentActionClick = {
    type: "click";
    payload: {
        selector: string;
    };
};
export type AgentActionEnterText = {
    type: "enterText";
    payload: {
        selector: string;
        text: string;
    };
};
export type UIActionMessage = UIActionToolCallMessage | UIActionPromptMessage | UIActionLinkMessage | UIActionIntentMessage | UIActionNotificationMessage | UIActionResizeMessage;
export type AgentActionMessage = AgentActionListTools | AgentActionCallTool | AgentActionQueryDOM | AgentActionClick | AgentActionEnterText;

UIMessenger client

The UIMessenger provides a clean interface for the embedded application to communicate with the host.

If using the UIMessenger client directly, then you have access to helpful methods like this:

const u = await initUIMessenger()
 
// Returns the data received during the handshake.
const data = u.getRenderData()
 
// Emits a "link" message to the client
u.link("https://www.example.com")
 
// Emits a "tool" message to force a tool call (of course this only works if respected by the client)
u.tool("sometool", params)
 
// And more!

Additionally, component resizing is handled automatically without any input from the developer.

useUIMessenger hook

This is sugar around the UIMessenger client that you can use in react.

If you call `useUIMessenger, then a UIMessenger client is automatically created, and you get access to data and all key UIMessenger client methods once the handshake completes.

const {data, link, tool} = useUIMessenger()

Note that the first render will have null data even if you expect a value in data, but all of the other methods exist, and will internally block until the handshake completes.

Fractal DocumentationPlatform Overview