RFC: Custom block plugin system

[tswsl1989]

This is just a proof of concept to try and get the ball rolling. The render definition for wool has been moved out of textures.py into a seperate file, which you can then add back in the config file using customblocks = ['wool']. This tries to render red wool using a custom texture, which I placed in /tmp/wool.png - you can download the file I used from http://sucs.org/~tswsl1989/texture-test.png

If you try this, then the texture will be visible, but very dark and hard to distinguish. I'm not sure if this darkening is the issue with the other wool colours.

Suggestions/comments?

[agrif]

Off-hand I can't think of why it would be darkening the wool, but it's not so weird that it's not possible.

As for the way the blocks are loaded, this is fine. I think it should be possible to load definitions from an explicit path, as well as by searching for them in the installed overviewer directory.

I've been working on a new Overviewer render backend that will require an overhaul of textures.py, but whatever replaces it will still be a single file containing block definitions. It'll likely still be a python file that uses something like @material, so this is still a good discussion to have now.

At the moment, because of how @material works, it's not possible to have per-render custom blocks. That'll be fixed, too. :D

[tswsl1989]

Loading from an explicit path should be possible as this is written at the moment (settingsValidators.py!236) - I admit that I haven't tested that bit though.

[Darius590]

Another thing to consider when looking at this is that some blocks need a custom drawing method (eg. engines,pipes).

I want to leave this here in case someone is looking for it and wants to use it.

Branch for Tekkit Classic 3.1.x: https://github.com/skeight/Minecraft-Overviewer/tree/tekkit
Screenshot: http://i.imgur.com/BYfmpTE.jpg

Branch for Tekkit Lite 0.5.7: https://github.com/skeight/Minecraft-Overviewer/tree/tekkit-lite
Screenshot: http://i.imgur.com/tcfDHcH.jpg

Branch for Hack/Mine 0.6.4.2: https://github.com/skeight/Minecraft-Overviewer/tree/hackslashmine
Screenshot: http://i.imgur.com/yTiOd8i.jpg

[0sanity]

Would be best to load all the material settings from a config file.

[CounterPillow]

@0sanity Yes, that's what we're kinda thinking of as well. Not necessarily the same config file as you define your render settings in, but some kind of modular plugin system.

[cicloid]

Is the ball still rolling? Is it dependent on OIL?

[tswsl1989]

Still rolling, but slowly.
It was decided that this was best left until OIL was ready. The mechanism itself isn't strictly speaking dependent on OIL, but the way the definitions would need to be implemented is. Unfortunately OIL hasn't progressed as fast as had been hoped.

[cicloid]

Ok,

Then, how could I contribute to that matter? In a way that doesn't enter into conflict with future development of OIL.

[agrif]

The great bulk of the work is actually making block definitions, not really the interface for registering them. Until the format used in OIL is set, any definitions written would probably need to be re-written.

I've been dragging on this, and I apologize. I'm hoping to finish that part up at least this week but historically such hopes have rarely been fulfilled.

[danaketh]

So, does this work? I really don't mind writing definitions myself (and fixing them in case of any changes) as long as the main code that handles them works.

You shouldn't be stuck on 3rd-party problem (if the OIL is a 3rd-party problem). Provide a working interface and let others worry about definitions.

[tswsl1989]

OIL is an internal issue. The issue with the code presented above was that it wasn't quite a working interface - there was an issue with custom defined textures appearing far too dark that I wasn't able to chase down at the time.

If I rebased this onto master, it would probably work well enough for people to try and fix that - but there's a good chance that things will have to change when OIL is finished and incorporated into Overviewer

[agrif]

an 8-month later update: new Minecraft snapshots include block meshes in the jar file itself, and this is probably part of a bigger move towards a block addition API. Best bet at the moment is to see how this format will finalize.

[Morketh]

Could the custom block plugin be able to read the mods folder and just grab textures from zip/jar files the same as the client? to me it seems an easier method of providing custom textures seeing as the mod author did that already to see them in game. Using this method i still see a problem with actual block rendering: "some blocks need a custom drawing method (eg. engines,pipes)" -- skeight above.
However i believe if the drawing method was in place adding textures from mod packs would at the very least provide the default textures for a modded game with little hassle. I spent 4 hours trying to figure out why my render on a modded map would crash looking for textures, which eventually lead me here. i am a python developer if i could contribute in some way please let me know

[CounterPillow]

Maps with modded blocks shouldn't crash while rendering. Overviewer just won't render those blocks. Open an issue with the traceback included, please, and anything special about your setup.

As for fetching the textures automatically: That's not possible, as far as I know, as every mod does whatever they want to do, as there is no unified API as of yet.

[Morketh]

thank you for the fast fast response this has helped narrow down the issue of a missing PNG from the texture pack i was using. i have not re ran with a new texture pack how ever i think im just going to upload the client jar to the server to fix it. CounterPillow above mentioned Overviewer wont render a mod block is this b/c the block is unrecognized or b/c it doesnt know where to look for textures?

[CounterPillow]

because the block is unrecognised.

[Morketh]

Would it be possible to hook into Forge in someway (yes i realize this was written in Python and Forge in Java) i was looking at the way mods are loaded using Forge and it seems there is a Standard Block Registration API
" Blocks need to be registered to Forge. To do so, use GameRegistry.registerBlock(Block block, String internalName). Add this to the load method. " ~ Forge Wiki

GameRegistry.registerBlock(genericDirt, "genericDirt");

now im not really familiar with this sort of in-depth programming and im just kind of throwing ideas on the page but to me it seems like it should be easy (again this might be extremely hard) to define a way to "define" a block (and its textures) based on the way that Forge Loads those blocks (again falling back to the above post were i had mentioned they got loaded into the game and are visible on the client side). im really kind of stepping out of my normal field here to see what i can learn about this. Now correct me if im straying a bit far into the brush here but wouldnt a bridge help in being able to load these pre-existing APIs that come with Forge? seems if you were able to hook into that mod loader and ask for texture paths (based on the mods location) and then figure out what a block was (based on the way forge constructs the blocks) it would be a giant leap in the direction of mod support i see a lot of talk about waiting for an API, why not use the one that's here?

EDIT: grammer

[CounterPillow]

Hi. Yes, block seems to have a standardised way of defining blocks, however, there are a couple of problems here. First off, the current block drawing code uses hand-written routines for constructing blocks for each id, with only some generalisation going on. This means that you can't throw arbitrary data at it and have it draw that. However, OIL will solve this™.

In the meantime, the official Mojang block model definitions have come out in 1.8 in some parts, but we still don't use it in OIL, mainly because nobody put in the effort so far to make it use the block model stuff. Nobody has touched the code in a while, mainly because they didn't feel any interest in writing that code. Most Overviewer contributors don't play Minecraft anymore.

And herein also lies the problem with the suggested Forge support: None of us have ever used Forge, and likely won't in the near future (though I can't speak for anyone else). Minecraft modding has taken a bit of a tumble with 1.8 and some stuff surrounding that release, and it all seems very up in the air at the moment.

From the technical side, just quickly glancing over the bridge stuff I don't think that's the right approach, since you wouldn't want Overviewer to start your Minecraft server. I think the more appropriate solution would be writing a Forge plugin that exports the block data Forge handles in some way for us to use, but all that requires someone to do that work, and the master branch is too unsuited for that sort of stuff while the oil (or is it oil-py3k?) branch is too far from complete to reduce the amount of outstanding work significantly.

Sorry :(

[Morketh]

Well that line of reasoning really gave me something to think about. Whats the status of Oil? Percent done, todo list, whats been done so far? i like your idea of having a forge side to Overviewer ill have to roll that idea around a bit and see what i can come up with. that seems like a more a modular solution seeing as forge keeps a dictionary of different blocks,items,liquids.what section of overviewer actualy handles the blocks? (the hand written parts you mentioned) maybe that will help me see whats going on in engine so i can toss a few wrenches at it

[CounterPillow]

The imaging code and rendering code of OIL is "done" as far as I know. It does not read block models from Minecraft as far as I'm aware, and a lot of the special case handling for the data values is not implemented. Furthermore, it's still missing other parts of a general rewrite we wanted to do, such as switching away from Google maps, a better POI system, a new config file format and so on.

The currently handwritten block definition parts of Overviewer are in the infamous textures.py file in the overviewer_core directory. Furthermore, some special case handling in terms of lighting is done in the render primitives C code which can be found in the src directory.

[3nt3]

did anything happen to this issue?

[tswsl1989]

did anything happen to this issue?

Not really. There were some odd issues with this proof of concept and since then the project has seen a lot of improvements.
The basic approach should still be viable if anyone wants to pick it up, but it's not likely to be something I do at this point I'm afraid.