Major Subsystem Rewrite (Reconciler Mk5)

[LPGhatguy]

This PR reimplements most of the big chunks in Rojo to shift their philosophy, enable some new features, and make it easier to understand and edit the codebase in the future.

• Closes Localization table column order isn't consistent #173: This was a simple fix that we could land in master too if we wanted to.
• Closes Rojo drops Ref properties #142: Refs are preserved by the new snapshot system correctly now!
• Closes .model.json Migration Guide #115: The naming behavior of .model.json files has reverted to match 0.4.x, and Name is now optional. This change was landed in master for testing ease.
• Closes Batched change events #96: The plugin protocol is being rewritten from scratch with batching from the start.
• Closes Support nested projects #95: Nested projects work as part of a general philosophy revamp in Rojo.
• Closes Automatically reload project file #10: Similar to Support nested projects #95, we got this for free with other changes.

Remaining Work

• Project format parity
  • init.lua, init.server.lua, and init.client.lua
  • foo.meta.json
  • init.meta.json
  • Project metadata like $ignoreUnknownInstances
• Revisit ImfsEntry API, potentially using generational_arena instead of spamming PathBuf
• Rebuild file watcher on top of new IMFS
• Plumb metadata through SnapshotMiddleware
• Build patch format to send over network
• Rebuild RbxSession and web server with new project lifecycle
• Update plugin with new protocol revision
• Write functional tests for rojo serve?
• Write benchmarks on master, port to this branch
• Delete old code

Philosophy Change

Instead of reading the project structure out of the project file at initialization time, the project file is discovered organically through the snapshot middleware! This has a few awesome side effects:

• Project auto-reloading: we just snapshot the entire project over again
• Nested projects: the top-level project and nested projects work the same
• Project files no longer necessary: rojo build can now operate on any file that Rojo can turn into an instance!
  • Convert a model from rbxm to rbxmx: rojo build input.rbxm -o output.rbxmx
  • Turn a Lua file into a model: rojo build main.lua -o MyPlugin.rbxmx

In-memory filesystem (IMFS)

The in-memory filesystem was rewritten to fetch files lazily instead of recursively reading in all files. I saw this as a prerequisite for nested projects so that we would avoid reading in lots of files accidentally.

All methods on the IMFS require exclusive mutable access, which makes some operations awkward since even reading can result in mutation. There might be a slightly cleaner approach involving interior mutability, possibly wrapping the data storage itself in an Rc or Arc.

Snapshot Subsystem

The snapshot subsystem was rewritten in order to break apart the code that generates patches and the code that applies them, which isolates the different forms of complexity.

I'm considering making applying a patch return a different kind of patch that actually gets sent over the network that would include ID information on instances that were created. That information is critical for correctly applying patches in the Rojo plugin!

Snapshot Middleware

The giant ball of mud in rbx_snapshot.rs for determining how to turn files into instances was broken apart into a folder of "snapshot middleware". A middleware is a struct that implements the SnapshotMiddleware trait. Examples of middleware are support for .lua scripts, support for .rbxm models, or a middleware that implements all user plugins!

The middleware trait defines how to turn in-memory files into instances, as well as instances to file snapshots: two-way sync! There's still work to do in adding the additional metadata context needed for live-syncing.

I'm not sure if generating snapshots of files is tractable. There is a single way to turn files into instances, but many different ways to turn instances back into files. While snapshots work well for the former case, it's possible that they're not enough (even with extra context) for two-way sync and we might need to revise this a bit.

Differences from Reconciler Mk4 (previous experiment)

This approach is the successor to #198. In this branch, the goal is to be less pedantic about child ordering than the reconciler-mk4 branch while still trying to accomplish the same overall goal.

The branch does not currently preserve child order across mutations. We can apply a similar strategy to this branch as we did to the current master branch if it's important.

[LPGhatguy]

0.5.0 is released, so now this can go into master and become the base for a new lineage of Rojo releases!