Skip to content

adding compress mode and checksum, as also split of symbol #59

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/advanced/objectlink/compress.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
sidebar_position: 8
---

# Compress Mode

In the compress mode the object and member identifier are replaced with unique integer values. This will done automatically by the code generator, if enabled. The compress mode will shorten the message size as also reduce the time to compare identifiers on client and service side. For example a property change message will be reduced from 1 integer, 2 strings and 1 json value to 3 integers and 1 json value.

```
[ PROPERTY_CHANGE, "org.demos.Echo", "message", "foo"]
// 1 is the object id of "org.demos.Echo"
// 2 is the member id of "message"
[ PROPERTY_CHANGE, 1, 2, "foo"]
Copy link
Contributor

@w4bremer w4bremer Sep 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify this, will it be really integers or will it be integers wrapped in strings?
[ PROPERTY_CHANGE, 1, 2, "foo"] or [ PROPERTY_CHANGE, "1", "2", "foo"]

Depending on the answer we should also adapt the protocol to use integer based functions params.
Otherwise we can just use one method for both modes.

```

The generated code will use the integer values for the object and member identifiers. To make this work and more reliable a checksum on a module level is introduced. The checksum is a hash value of the module content and will be used to verify the remote object is the same as the local object, to avoid using the wrong object or object version.

The link message will carry an additional checksum value.

```
[ LINK, 1, CHECKSUM ]
```

There is no handshake messages, to ensure both client and services are in compress mode, this needs to be ensured during the deployment. If the client and service are not in the same mode, the link will fail, as the object identifier will not match.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be much work to extend the link message to also sync the mode?
Bit more work now, but less headache during deployment because there would be a clear error message.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exchanging the protocol version would also help reduce problems.

30 changes: 14 additions & 16 deletions docs/advanced/objectlink/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,25 +46,23 @@ The client and server needs to know their transport and encoding in advance. The

## Message Formats

| Direction | Message | Value | Format |
| --------- | ----------------- | ----- | -------------------------------------- |
| `->` | `LINK` | 10 | [ MsgType, ObjectId ] |
| `<-` | `INIT` | 11 | [ MsgType, ObjectId, Dict ] |
| `->` | `UNLINK` | 12 | [ MsgType, ObjectId ] |
| `->` | `SET_PROPERTY` | 20 | [ MsgType, PropertyId, Value ] |
| `<-` | `PROPERTY_CHANGE` | 21 | [ MsgType, ObjectId, Value ] |
| `->` | `INVOKE` | 30 | [ MsgType, RequestID, MethodId, Args ] |
| `<-` | `INVOKE_REPLY` | 31 | [ MsgType, RequestID, Value ] |
| `<-` | `SIGNAL` | 40 | [ MsgType, SignalId, Args ] |
| `<-` | `ERROR` | 50 | [ MsgType, MsgType, RequestID, Error ] |
| Direction | Message | Value | Format |
| --------- | ----------------- | ----- | ------------------------------------------------ |
| `->` | `LINK` | 10 | [ MsgType, ObjectId ] |
| `<-` | `INIT` | 11 | [ MsgType, ObjectId, KWargs ] |
| `->` | `UNLINK` | 12 | [ MsgType, ObjectId ] |
| `->` | `SET_PROPERTY` | 20 | [ MsgType, ObjectId, MemberId, Value ] |
| `<-` | `PROPERTY_CHANGE` | 21 | [ MsgType, ObjectId, MemberId, Value ] |
| `->` | `INVOKE` | 30 | [ MsgType, RequestID, ObjectId, MemberId, Args ] |
| `<-` | `INVOKE_REPLY` | 31 | [ MsgType, RequestID, Value ] |
| `<-` | `SIGNAL` | 40 | [ MsgType, ObjectId, MemberId, Args ] |
| `<-` | `ERROR` | 50 | [ MsgType, MsgType, RequestID, Error ] |

- `MsgType`: integer value of message type
- `ObjectId`: a string identifying the resource as module and object name (e.g. `"demo.Calc"`)
- `Dict`: A JSON dictionary, e.g. `{ "count": 0}`
- `ObjectId`: a string identifying the object (e.g. `"demo.Calc"`)
- `MemberId`: a string identifying the member of the object (e.g. `"count"`)
- `KWArgs`: A JSON dictionary, e.g. `{ "count": 0}`
- `Args`: A JSON array, e.g. `[ 1, 2 ]`
- `PropertyId`: A ObjectID with a property path (e.g. `"demo.Calc/count"`)
- `Value`: Any valid JSON value including JSON arrays or objects
- `MethodId`: A ObjectID with a method path (e.g. `"demo.Calc/increment"`)
- `RequestId`: A unique integer value identifying the request during the connection. Typically a value incremented by one on each request and starting by 1 and then reset to 1 by max value.
- `SignalId`: A ObjectID with a signal path (e.g. `"demo.Calc/shutdown"`)
- `Error`: A string describing the error
13 changes: 9 additions & 4 deletions docs/advanced/objectlink/lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,20 @@ sidebar_position: 3

# Lifecycle

To use a remote object the object needs to be linked first using the object name. The `LINK` request will be answered with a `INIT` message to initially populate the local properties from the remote object and subscribe to property changes and signals. After the linking the local object is fully usable.
To use a remote object the object needs to be linked first using the object name. The `LINK` request will be answered with a `INIT` message to initially populate the local properties from the remote object and subscribe to property changes and signals. After the linking the local object is fully usable.

The check sum is used to verify the remote object is the same as the local object. If the checksum is not the same the link will fail. A checksum should be automatically generated by code generator based on the relevant content of the API module (e.g. interfaces, properties, signals, methods, structs, struct fields, enums, enum values, etc.).

In case the checksum does not match the server will return an error message.

A link message is send by the client to link to a remote object.

```js
--> [ LINK, "org.demos.Echo"]
--> [ LINK, "org.demos.Echo", CHECKSUM, ]
```

A link message is answered with an init message with the initial properties, or an error message, in case of failure.

```js
<-- [ INIT, 'org.demos.Echo', { message: "hello" } ]
```
Expand All @@ -25,11 +30,11 @@ To release the resources on the server side the object can also be unlinked, thi

## Sequence Diagram

To link a local object to a remote object we need to send a link message.
To link a local object to a remote object we need to send a link message.

```mermaid
sequenceDiagram
Sink->Source: [LINK:int, ObjectId:string]
Sink->Source: [LINK:int, ObjectId:string, Checksum:string]
Source->Sink: [INIT:int, ObjectId:string, Properties:jsonObject]
Sink->Source: [UNLINK:int, ObjectId:string]
```
23 changes: 10 additions & 13 deletions docs/advanced/objectlink/methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,14 @@
sidebar_position: 4
---

# Method Invokation
# Method Invocation

A remote invokation invokes asynchronously a method on a remote object and returns the result.
A remote invocation invokes asynchronously a method on a remote object and returns the result.

## ApiGear Object Model

To model methods in ApiGear you define an interface with operations.


```yaml
name: org.demos

Expand All @@ -19,14 +18,13 @@ interfaces:
operations:
- name: say
params:
- name: msg
type: string
- name: msg
type: string
type: string
```

The operations will be generated as methods of the object. This will look simplified like this.


```js
// org.demos.js
class Echo {
Expand All @@ -36,7 +34,7 @@ class Echo {
const echo = new Echo()
console.log(echo.say("hello"))
$> hello
````
```

## Protocol Flow

Expand All @@ -45,22 +43,21 @@ To invoke remote method an method name and the method arguments must be specifie
The local object sends an `INVOKE` message to the remote object using a request id, the method name and method arguments.

```js
--> [ INVOKE, 1, "org.demos.Echo/say", ["echo"]]
--> [ INVOKE, 1, "org.demos.Echo", "say", ["echo"]]
```

The remote object executes the method and returns the reply or an error message in case of failure.
The remote object executes the method and returns the reply or an error message in case of failure.

```js
<-- [ INVOKE_REPLY, 1, "echo"]
```


## Sequence Diagram

After an object is linked remote methods can be called.

```mermaid
sequenceDiagram
Sink->Source: [INVOKE:int, RequestId:int, MethodId:string, Args:jsonArray]
Source->Sink: [INVOKE_REPLY:int, RequestId:int, MethodId:string, Value:Json]
```
Sink->Source: [INVOKE:int, RequestId:int, ObjectId:string, MemberId:string, Args:jsonArray]
Source->Sink: [INVOKE_REPLY:int, RequestId:int, Value:Json]
```
24 changes: 10 additions & 14 deletions docs/advanced/objectlink/properties.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,18 +27,18 @@ The resulting simplified typescript code could look like this.
```js
// org.demos.js
class Echo {
message: string = ""
message: string = "";
}
const echo = new Echo()
echo.message = "foo"
const echo = new Echo();
echo.message = "foo";
```

## Protocol Flow

First the local object needs to be linked to a remote object.

```js
--> [ LINK, "org.demos.Echo"]
--> [ LINK, "org.demos.Echo", CHECKSUM]
```

Now the local object receive initial property list, which is automatically send after the link message.
Expand All @@ -49,28 +49,24 @@ Now the local object receive initial property list, which is automatically send

After the init message out local object is fully populated and all properties have valid values.



When a property is changed on the local object, for example from "hello" to "foo", a `SET_PROPERTY` message is send.


```js
--> [ SET_PROPERTY, "org.demos.Echo/message", "foo"]
--> [ SET_PROPERTY, "org.demos.Echo", "message", "foo"]
```

The remote object will then set the property and notify all linked objects about the changes using a `PROPERTY_CHANGE` message, including the original sender.

```js
<-- [ PROPERTY_CHANGE, "org.demos.Echo/message", "foo"]
<-- [ PROPERTY_CHANGE, "org.demos.Echo", "message", "foo"]
```


## Sequence Diagram

After an object is linked propertie will be synced across all linked clients.
After an object is linked properties will be synced across all linked clients.

```mermaid
sequenceDiagram
Sink->Source: [SET_PROPERTY:int, PropertyId:string, Value:json]
Source->Sink: [PROPERTY_CHANGED:int, PropertyId:string, Value:json]
```
Sink->Source: [SET_PROPERTY:int, ObjectId:string, MemberId:string, Value:json]
Source->Sink: [PROPERTY_CHANGED:int, ObjectId:string, MemberId:string, Value:json]
```
25 changes: 10 additions & 15 deletions docs/advanced/objectlink/signals.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,12 @@ sidebar_position: 6

# Server Side Signals

Signals re used to notify the local object about changes on the remote object.
Signals re used to notify the local object about changes on the remote object.

## ApiGear Object Model

Signals can be modeled using ApiGear as signals of an interface.


```yaml
name: org.demos

Expand All @@ -19,15 +18,16 @@ interfaces:
signals:
- name: shutdown
params:
- name: timeout
type: int
- name: timeout
type: int
```

The resulting code will look somehow like this and most often will require a lambda function to be used for the notification.
The resulting code will look somehow like this and most often will require a lambda function to be used for the notification.

```js
// org.demos.js
class Echo {
onShutdown(callback)
onShutdown(callback)
}
const echo = new Echo()
echo.onShutdown( (timeout) => {
Expand All @@ -39,32 +39,27 @@ echo.onShutdown( (timeout) => {

To receive signals the local object needs to be linked to the remote object first.


```js
--> [ LINK, "org.demos.Echo"]
--> [ LINK, "org.demos.Echo", CHECKSUM]
```


Then the remote object can send at any time signals to the linked client objects and notify them on changes.

```js
<-- [ SIGNAL, "org.demos.Echo/shutdown", [10]]
<-- [ SIGNAL, "org.demos.Echo", "shutdown", [ 10 ] ]
```


To stop receiving signals, just unlink the remote object.


```js
--> [ UNLINK, "org.demos.Echo"]
```


## Sequence Diagram

After an object is linked server side signals will be send.

```mermaid
sequenceDiagram
Source->Sink: [SIGNAL:int, SignalId:string, Args:jsonArray]
```
Source->Sink: [SIGNAL:int, ObjectId:string, MemberId:string, Args:jsonArray]
```