Assuming we are discussing server-side LOD texturing, I would recommend sending 16-bit colors. When you combine biome color variants, and especially if you are downsampling, the number of color combinations is going to be too high to comfortably index. Also keep in mind that a.) the overall color of a block is determined by the client's texture pack, and b.) the client and server won't necessarily have the same block IDs. So that'll be fun.
@4HeadTiger: What I meant with my line about the 1:1 interaction was that only the boundary-changing edges of a body of water need to actively query the chunk. Think about the way water works now. When a block is at the edge of a flow going down a hill, it will expand based on the surrounding blocks. If the water block's neighbor rules have already been satisfied, the block is stable and will not change again unless a foreign block interacts with it. This can be applied to the dynamic model as well. Worse case, you'll end up with maybe a thousand active water block spanning a handful of chunks which are actively deciding whether to spread or trigger a block update. Everything behind them are stable and can be handled with the abstract model. If that's the numbers we end up dealing with, that's manageable without going to extra lengths to avoid it.
A special case, though, is waterfalls and water moving down slopes. Over a window of time, their flow rate should remain constant if their source and sink do, but depending on the flow rate, it may not look static every tick. I'm not completely sure what to do about them, but I feel like they shouldn't require direct chunk access if no one is in the area.
Edit:
With runoff, I'm still concerned about water collecting in unwanted places. In caves, for instance, since the things weren't cut out with water, they're not going to have a proper drainage route, and if a player builds in a dry valley, that valley is going to fill up over time unless we can drain water through dirt and rock. I'm just not sure it's safe to let the server blindly distribute water without some idea of where is and isn't a good place for it to end up.
We need to have lots of integer fluid levels per block. 8 is not enough. 16 is not enough. In my mod, I'm using a grand total of 16384 levels, 2^14, fluid levels per block.
...
Ideally, this data would go in the same place as the rest of the data structure, such that we can retrieve it very easily by merely testing a single bit in a block ID (aka, water uses the bit 0x01000000 | fluid_level (0 to 16384), then to test we just see if storedID & 0x01000000 == 1 => fluid_level = storedID & 0x3FFF)
...
Additional changes: do something similar for all solid blocks (and blocks with other properties) such that the ID is technically still stored in 4096 block IDs or whatever, but we can query various properties much MUCH much faster. What we end up with is a table of bits which tell us more about a block without having to do many queries with the block itself;
Solid = 0x0001 0000
Water = 0x0100 0000
Lava = 0x0200 0000
etc
This part concerns me a bit. I'm not sure how this is intended to mesh with your concept of chunk-level statistics, but it sounds like you would still be handling certain liquid interactions at a 1:1 scale. I don't doubt that this is necessary in order for a stream of water to travel over new ground, but might we be able to make a distinction between water blocks actively altering the boundaries of a body of water and those which are merely flowing internally?
Edit: wait, I think that is sort of what you are suggesting; separating the distinct blocks a player will see move around from the macro-scale water mechanisms.
Additionally, I would model the water collision layer a bit more simply. You have your solid blocks, your air blocks, blocks with will trigger important chunk updates (e.g. redstone), and blocks with unimportant chunk updates (e.g. grass). If a block needs an important chunk update, it's probably not worth it to further guess at its behavior without simply loading the chunk up and interacting with it.
Distributions have averages. This means we can quickly estimate the content of an ocean by merely counting the number of ocean chunks.
Unfortunately, life isn't this easy, so just estimating total fluid in an area might not always be good enough. I'd suggest using 17 integers per cube. One integer stores the total fluid in that cube(just like how a block contains a certain amount of fluid), with the other 16 representing the amount of fluid at each layer in the chunk (call this a histogram for simplicity). We will also need to figure out a way of recognizing how histograms are connected, and we may also need a similar system for recognizing solid blocks at a layer
I can get behind this idea, but I'm not sure the chunk layer is the level I would store it. If it were me, I would treat a contiguous body of water as a single abstract object (with special handling for oceans. I would also try to subdivide that body of water into convex hulls and graph the internal flow rate between them by using the surface area of the hull borders. That won't solve the complications of territory altering water flow, but it should scale well once a body of water has stabilized.
Otherwise, you'll be running into the issue of two isolated streams of water occupying the same chunk when we zoom back out.
Now we need to design the actual fluid simulators, since hydromechanics is no fun if the water doesn't move
Water, magic as it is, likes to flow down. At the block scale, a fluid block will try to move fluid to the block below, then to the sides. At a histogram scale, a higher histogram will attempt to move fluid down, and then to the sides (iff the adjacent histograms are connected). On a chunk scale, a chunk will try to move fluid into any lower chunk, then into an any adjacent chunk with less water. Wonderfully simple. Using some threading, scheduling tricks, etc, we can make everything happen without causing lag.
...
To jump between the "collection of blocks" and the "simplified representation", all we have to do is manage the fluid counters whenever the fluid of a block changes.
Note that when dealing with chunks, we can perform the updates infrequently, moving large quantities of fluid in each step, meaning that the overall computational cost is going to be relatively low even when dealing with a HUUUGE area.
As long as the water channels are stable, this sounds about right. That's what you meant, right?
Finally (finally lol) we get to the meaty stuff - moving water from A to B, making rivers flow endlessly, etcetera. The first thing is to ensure that oceans don't overflow. To do this, we simply remove water from random ocean chunks at random times, and add it to our global counter (or a regional counter, or whatever). We can also make water evaporate and so on, with the lost fluid all going into the "reserve counter". We now use this counter to put water down at the start of rivers, and if we want; to make it rain, affect the cloud cover in certain places, and so on. All sorts of fun things can be done with the spare water
How exactly one ensures that the water is returned to the correct place needs some more thought. It may be that once the terrain is generated, we determine regions where water will be deposited, then explicitly gouge the aforementioned tributaries into the terrain such that most of the water ends up in the main river. There may be other approaches. I don't really know as I haven't really spent much time thinking about the specifics yet.
If we are guaranteeing that a fixed amount of water exists in the hydrosphere, wouldn't it just be more busy work to simulate precipitation when we know what the results ought to be? I mean, unless you plan on simulating droughts. My concern is that simulating things like runoff could be expensive to handle procedurally, and if it results in water appearing in places not originally intended to house water, it might end up causing trouble for players who might not be happy to discover their mine has transformed into an underground swimming pool over the course of a month.
In any case, I'm of the strong opinion that water propagation should not be handled without direct chunk interaction if it is unclear what the results are going to be. That's just asking for headaches. If stable flows can be abstracted to a less expensive simulation, I don't think the remaining roaming flows will be prohibitively hard hard on the budget, especially if we can tell beforehand whether a chunk update is going to be invoked.
In terms of large surface pre-generation... unless there is some bizarre aspect of the game that forces you to generate the entire surface... What if you query arbitrary parts of the noise maps, then use the results to estimate a rough mesh? It's basically octree culling, but instead of culling existing nodes, you use a tree to generate only the necessary nodes.
What it means is that in 'step 1', you can generate X points distributed across the surface of an N km² planet. Now the cube-planet can be represented with roughly the correct colors in roughly the right places and so on. In step 2, you generate another Y points on the surface and refine the mesh a little bit, and so on. Threading these steps would even let you generate the player's immediate surroundings very very quickly and fluently, while gradually constructing and improving the distant terrain towards some predefined threshold without locking the player in a loading screen or whatever.
There actually is one bizarre aspect of terrain loading, and that's decorations. Once you generate your volumetric model and convert it to biomes, you have to decorate it with ores and other block variants, followed by vegetation and macro structures like villages, ravines, and temples. At 5km, these things aren't small enough to simply ignore, at least I don't think so, and vegetation is by far the worst. At present, I don't have a good way of mapping trees, but it would be very obvious if they suddenly decided to show up as you got closer.
With the hydrosphere, bodies of water will also need to be added after the initial terrain is analyzed.
Wow, you guys really didn't like my simplified cube representation idea.
Nah, this is just haggling at this point. But it doesn't hurt to keep this stuff in mind so we don't actively sabotage it getting added down the road.
The way I see it is this. Right now, outside of the loaded cubes, nothing at all is rendered. I just want to render something there. I don't actually care much about what it looks like right now because rendering something has to be better than rendering nothing. To start with, we should render the simplest possible thing there. Once that's actually working, then we can think about how to make it fancier. If we start with something too complicated on the first try, it will never happen at all.
As for lighting, maybe we can model scattering using some fancy model (and I think it would be really fun to try), but I think I'm going to table that problem for another day. Let's just get the current implementation of cubic chunks out there, raise the build limit to ~8 mil, have a nice drink, and celebrate a job well done.
I agree with this except for the lighting model. Considering that this has always been the stumbling block in every cubic chunks attempt, if we don't come up with a practical solution this mod will end up dead in the water regardless of success on any other front.
I would also recommend holding off messing with occlusion culling until 1.8, since it will be part of the pipeline by that point.
Dynamic fluids would be very difficult I can't recommend it. Btw, here is a mod where I am totally doing just that and it's freaking awesome.
Oh, you're such a tease.
Alright I'm sold, hydrodynamics is now in the definitely column.
In seriousness, though, another important element of rivers you may not have considered is how terrain is generated. The world is generated from layered Perlin noise, obviously, but unless there are an infinite number of octaves, the pattern begins looking uniform when you pull far enough back. The Island continents can't really get much bigger than they are now unless we add another octave or two to the noise function (not a bad idea, really, since the features all start to look miniature with an upped draw distance).
But even if we got those bigger, more realistic terrain features, we still can't expect to scale up to IRL sized features and phsyics without getting unwieldy for the player. For instance, the map of Skyrim is only 4 miles from one end to the other, but it feels expansive because you are on foot and there's lots of detours along the way. With that in mind, we need to have a way to handle fluid dynamics that still works on a much smaller scale while keeping things in equilibrium.
My proposal is scaling the virtual capacity of a body of water disproportionately to its physical volume. For instance, a small pool with a hundred blocks of water is treated as containing 100 blocks of water. However, a lake a hundred meters across may be handled like it contains 10 or 100 times as much water as it actually does. Oceans would effectively be infinite.
Combine that with rain and tributaries from mountain runoff (a reasonable feature with a larger terrain scale) and it would be possible to keep the water levels stable over a period of time. Plus, during the initial terrain generation it should be possible to only place lakes where equilibrium is achievable. If we can then rely on the assumption that equilibrium is the natural state of rivers and lakes without any direct player intervention, that should make life a little bit easier for the macro-scale simulation.
Then there's Futurecraft, which is the whole reason I'm investing research into cubic chunks. In Futurecraft, you live on cubic planets which are ~5km to a side (the whole surface has to be pre-generated, and that's about as large as can be reasonably handled), If we came up with some way to simulate a fully functional (or at least functional-looking) hydrosphere down at that scale, that would just be the best thing ever.
Whether we actually implement it or not, I think it is feature definitely worth humoring, and trying to accommodate it as we go about reworking other aspects definitely going in, like terrain generation.
We don't actually need to compress the representation of multiple light sources into a few coefficients since for sunlight, there's only one direction we care about. Unless you want to represent scattered light using functions over S^2, but then we're back to expensive iterative things like global illumination. Maybe if we choose the right network of light "sources" (ie a set of coefficients at a point) by computing some kind of network/graph over the terrain, we could keep lighting efficient, but that sounds like a non-trivial problem too.
On the other hand, representing all lighting as a graph of coefficients at points could mean we could do away with saving explicit light values at each block. That could improve cube read/write/transfer times significantly. Then lighting for each block could be calculated on demand. Btw, my "cube" term means a 16x16x16 array of blocks.
The intent isn't so much to support more realistic lighting (though I doubt anyone would complain if that was the end result). Rather, it's to get something, or anything really, to act as a surrogate for the "light shafts travel indefinitely and illuminate everything along the way" effect without resorting to querying individual blocks. What that effect is supposed to do is act as a surrogate for the idea that 1) light bounces off a surface and lights up the surrounding area (global illumination), and 2) scattered light from the sky and ground prevents anything not directly lit by the sun from getting particularly dark (macro-scale ambient occlusion).
If we ignored 2), we could just do the regular light diffusion at the point where the sunlight hits a surface. That would solve the scaling issue, but anywhere there was a sheer vertical drop, like cliff faces and mine shafts, would be left in complete darkness. On the moon, if the sun was perfectly overhead, this is exactly what would happen. But we live where there is a sky, so 2) needs to be taken into account in some way.
The reason for spherical harmonics isn't to represent direct sunlight, in fact you would explicitly ignore direct sunlight since it acts like a beam and doesn't diffuse. The thing it's representing is that directional light from the sky and the ground. The reason I say it's directional, even though it's in every direction, is due to the cases where it's partially occluded by a tree or a cave. The further underneath it you get, the darker things become, not because you are further away from the direct sunlight, but because you are less exposed to those ambient lights.
If we had a different way to approximate macro-scale ambient occlusion, we could use that instead. But say you're working out the lighting for a giant cave, maybe it's fully enclosed, or maybe we're near the mouth, it's tough to tell. In a case like that, it may end up being just as much effort to do "shortcuts" as it would be to just bite the bullet and record some real numbers.
I should probably also mention that I expected the light propagation to be either fully server-side or at least server-assisted. The results would still get transformed back into discrete 4-bit light levels for rendering and interaction (but this only matters within the region actively experiencing ticks, we could do whatever we wanted in scenery beyond that).
It might work, but it sounds like a HUGE departure from the existing lighting system. There's gotta be an easier way.
I have my doubts about that. This is the the defining point where everyone's attempts to-date have broken down, and is the thing spooking Mojang away from doing it themselves. No amount of adaptation to 1:1 light propagation will ever permit it to scale, so one way or another we'll have to go off the deep end if we want this. Not necessarily spherical harmonics, but the formal alternatives aren't especially entry-level, either.
Next, onto LOD rendering and occlusion culling. I think some of the latest 1.8 snapshots added support for vertex buffers. That (I think) should replace the call lists that were originally used for rendering and make things a lot more efficient. I don't want to spend too much time trying to work on rendering efficiency now if there are already fixes coming down the pipe.
VBOs have actually been supported for a really long time. If the GPU supports it (and realistically that's all of them), geometry is referenced from the stored buffer. If not, the polygon is given the draw command then and there. Either way, everything is getting compiled to display lists, which are immutable collections of optimized GL commands that get executed in bulk. Display lists are now depreciated, because they clash with the design philosophy of the programmable shader pipeline, but functionally they still work the way they always have and there's no advantage performance-wise to removing them.
What is interesting is there was a MogMiner tweet not too long ago saying they were going to remove all the fixed-function GL commands for 1.9. I'm not sure if that extends to the display lists, but if it does, that's going to create a major architectural rewrite of the graphics pipeline. I kind of hope they do, since it could be fantastically powerful boon for shader modding, but it could also wreak havoc on the programmatic way entities are rendered. Or maybe they'll support both methods for a while to ease the transition. We'll just have to wait and see what happens.
If we want to increase the render distance though, we definitely need some kind of LOD. Even if we try to brute force the rendering, we're going to end up with checkerboard objects in the distance because the blocks just get too small. LOD also goes hand-in-hand with the cube loading. Cubes have to be loaded somewhere to be rendered. If we only load far cubes on the server, then we can send some kind of less-detailed representation to the client, but what's the right representation to choose? It seems tempting to take advantage of the directional nature of far cubes (ie, a slow-moving player will only see a cube from essentially one viewpoint), but then we can't do that efficiently on a multi-player server without having the server do the rendering for the client. There needs to be some kind of direction-independent simplification for a cube. I don't really care about preserving blockiness so much, as long as we can just put up some color in the background of a viewport and have it look reasonably good.
Only loading the faces visible from the camera's direction has been proposed before, but it's almost more trouble than it's worth. With backface culling, polygons facing the wrong way (as determined by a right-hand rule in the vertex ordering) are completely eliminated before incurring a pixel fill cost, though the vertices still have to be transformed into screen coordinates before this can be determined. Assuming you are not swamping the command pipe, you could split up each chunk into 6 layers, one for each face, and draw 3 of them in most occasions, but even with an upper bound of 50% savings (which is unlikely), thanks to the law of cubes that would only let us extend our draw distance by another ~26% before we were back to where we started. The only way to put a real dent in the scaling factor is to get as close to O(n2) space complexity as possible. Anything less won't matter in the long run.
If you mean prerendering parallax backdrops, that can sort of work, but only if you are viewing it from roughly a single direction. If you circle too far around or fly overhead, the illusion breaks down.
As for occlusion culling, I was eventually planning to come up with some kind of data structure that could answer visibility queries efficiently. Then the server could only send cubes that were visible to a player (or close enough to be heard regardless of visibility). I haven't really started thinking about this much though.
Occlusion culling really needs its own post. I'll write something up regarding that this weekend.
Yup, you should work on the mod with us. =)
You'd come to regret that when I become swamped with work and disappear for 6 months. Again.
Oh wait, I just came up with an awesome idea for a LOD cube simplification. It's SOOO Minecrafty, it might just be perfect.
How about we just simplify a whole 16x16x16 cube into a single block?
We can pick the modal block in the cube to represent the cube. Mountains in the distance are going to look ridiculously blocky, and I kinda think that's going to be completely awesome. We could also store the cube-as-block things extremely efficiently in the terrain database so the server doesn't have to load full cubes all the time to do the simplification. We can just maintain the cube-as-block things when we update the full-resolution block terrain. Sending them to the client is going to be super easy. We can't get around the law of cubes, but sending n/16/16/16 blocks instead of n will go a long way.
Yeah, it's been done and it is pretty ugly. Some other blockworld engines have gotten away with 2x or even 4x scaling, but more than that and there's too much loss of information and it becomes distracting. If we are getting out to the point that we can use 1/16th the detail, a google earth style tiled texture projected onto a heightmapped mesh is a more traditional approach to game terrain and will look closer to the real thing than trying to retain blocks. It shouldn't be necessary, either, with the blockifying shader, but you were the one who brought it up.
Boy, sure wish I had been paying attention a month ago when all the cubic chunk mechanics was being discussed.
Oh well, better do it all at once, then.
=========== Introduction =============
Obviously, the problem with supporting cubic chunks isn't one of storage or encoding, but the implication that two things can and need to interact on a scale that you can't possibly hope to simulate. No matter how efficient and tightly optimized your per-tick code is, the Law of Cubes is going to make you her *****.
So what can we do about it? Well, if you look at other voxel engines (of the volumetric modelling variety, not the Minecraft block worlds), you'll notice that to the man, every implementation features very small voxels, to the point that individual voxels become indistinguishable from one another (except for Ken Silverman's Voxlap engine, but that one's a freak of the voxel world). Most of the time, the developer will go to great effort to hide the discreteness of the voxel structure. Because that detail is obscured, it opens opportunities for all manner of corner-cutting. Oct-tree storage (subdiving the grid only where there is a meaningful improvement in fidelity), google-maps style streaming detail, acceleration structures (mechanisms mixed into the data model to tell the render where it can take shortcuts, like large empty spaces. The entire discipline is about making voxel data smaller, faster, and more intelligent.
...Except for block worlds. Around here, we care that everything is sharply defined and cubic, and we also care about 1:1 detail, which is why we can't leverage all those state of the art techniques that make voxels actually a viable, scalable product. But what if we could? In an effort to play by the rules of the vanilla Minecraft physics engine, we are artificially making things harder on ourselves than they need to be. If you managed to get a 2km draw distance, you would find that individual blocks create a awful looking Moire pattern, since only one polygon can occupy a single screen-space pixel at a time (without a supersampling AA which is expensive doesn't really agree with Minecraft). So why should we have to draw blocks that way? If you make a 31-wide platform a mile up in the air, Minecraft says the shadow it casts would be completely dark in the center. Even if it were feasible, it would look ridiculous, so why are we trying to do that? Same goes for water, why should we consider water on a block-by-block basis if it becomes impossible to handle fluid dynamics on a large scale?
========== Lighting ===========
Minecraft lighting simulates a poor man's global illumination-the idea that rather than hard light and shadow, light bouncing off surfaces lights up everything nearby, making a smoothish gradient. The other principle tenant of global illumination is that the strength of light isn't a function of distance, but the arc-length of the source. If you have a nearby lightsouce and a giant wall of light in the distance, but both take up the same amount of your field of view, they are contributing the same amount of light to your position. The practical thing this means for us is that in terms of light contribution (excluding direct sunlight), the farther away a light source is-the larger it has to be in order to have any influence on a particular surface, meaning you can be working with lower and lower resolution models and getting the same effect. It would be difficult to tell whether you are inside or outside in a cave the size of a baseball stadium, but scaled down to a hut, it's not so difficult.
I mentioned sunlight didn't fit this model, and that's because the sun is so large and far away that its arc length doesn't change appreciably no matter where you are, so assuming you are not trying to simulate the sun moving through the sky, all you need to know is the heightmap to see if anything directly above you is occluding the sun to know whether you are in shadow, no big deal. But I also said before that it would be ridiculous for a small platform a mile up to be casting any appreciable shadow. The reason for this is that direct sunlight is not the only source of light. You also have ambient lighting. To keep thngs simple, ambient light is light coming from no place in particular. In our case, it's coming mostly from the scattered light in the sky and reflected light on the ground. If you are out in the open, it's the same no matter where you go. If you are under a tree, the leaves are blocking the sky so your only ambient light is coming from the horizon and reflected off the ground. In other words, when not directly lit by the sun, the amount of light hitting a block is the arc-area facing the sky, lit environment, or nearby point lights.
If you fully simulate the contribution of light from all these sources, and then recalculate again with those results, over and over until the new numbers stop changing appreciably, then you've just done global illumination, and the results will look hella good. On the other hand, it's also hella expensive, to the point that only recently has anyone even attempted doing it in realtime with the beefiest of GPUs. Additionally in the close range with block lights, and with direct sunlight, the current system isn't a terribly bad imitation. It's only those intermediate distances, where you're in the air, not next to any block light, but you're also not directly lit by sunlight, where the immitation starts breaking down. The lighting algorithm also spends a disproportionate amount of time lighting up these places that don't actually contribute anything. So why not scrap the idea that sunlit spaces need to radiate?
This next part is hard to explain, so I'm just going to give a rough outline for now. There is a complex system of equations in the world of mathematics called spherical harmonics, which are handy for representing directional lighting through a point in space. What it is is a set of equations which describe any arbitrary combination of directional magnitudes in 3D, translated to a set of coefficients. It's very similar to how you can recreate any sound by combining enough sine waves at the right frequency and amplitude. To apply lossy compression on a sound, you try to remove all the sine waves that don't significantly affect the result. Jpeg compression works more or less the same way, first converting color gradients to waves. The same applies to spherical harmonics. By adding or removing coefficients, you can increase or decrease the accuracy of the representation. What this means in practice is that we can represent light rays passing through a point from an infinite number of directions with a relative handful of parameters. Just don't go trying to simulate a laser pointer.
So now you can imagine a world filled with these things, one in the middle of each air block, each telling their neighbors how much light that vector is contributing. That takes care of getting light to travel in all directions, but it's still really expensive per point, and we're processing even more points than the old way. That's where the scaling rule comes into play. Light sources farther away need to be larger to contribute any meaningful light, so you can handle them at a lower resolution. As far as physical placement, this means larger open areas can be represented with fewer spheres. Out under the sky, they would be huge and paint with massive strokes, which is what you would expect from ambient light. We'd still use direct sunlight and point lighting, as well as scattering in the places where sunlight actually strikes (not simply passing through), but the big empty spaces are now vacuums that light simply passes through.
There are a number of considerations with this, on top of the math being fairly gnarly, like what arrangement and density of spheres gives us the best bang for our buck, and will this simulation work if there is a large difference in brightness between the sun and block lighting? Additionally, though it would be less noticeable, we'd need to simulate light propagation independently for block lights. Individual points of light die off pretty quickly, but in a big room full of point lights, the numerous little contributions add up and even-out the light levels (this is floating point math, so tiny contributions are recorded just fine so long as all the contributions are in that same range, which is what we want). With the right tweaking, and especially using the assumption that the sun rays are stationary, there's no reason this shouldn't be feasible, at least server-side. Distant changes are also less noticeable, so there is no reason long-distance light propagation has to be done all at once, either.
=========== Block Rendering ============
I won't go into great detail here, because this is part of my active research and there's too much to cover. Block geometry takes up a lot of space on a GPU. If memory serves, normal vertex stores world coordinates, texture coordinates, an additional texture coordinate for lighting, and a color for biome-sensitive blocks like grass, and a couple bytes of padding, multiplied by 4 vertices per face (and block faces never share vertices) adds up to 128 bytes per block face. That's a lot, and you run out fairly quickly, which is why the draw distance is so low. The worst part is, thanks to that law of cubes, every bit of range you increase the draw distance by increases the memory footprint geometrically, which no amount of culling or optimization is going to offset for long. What we need here, and what all those other voxel engines do, is implement level-of-detail.
The obvious problem behind LOD is that the block world environment wears its discrete details like a flag, and you can't simply reduce the geometry in the blocky parts without it being spectacularly obvious. If the texture doesn't change, you can join up adjacent block faces in the same plane to use fewer polygons. Nocte does this for Hexahedra, and it's been used by a couple other block world engines as well. Since lighting was being stored at each vertex, this can pose a problem when not every tile has its own vertices. One engine solved that by using a dense 3D texture of light values and married the light and geometry via shader (though a dense 3d lightmap is pretty heavy in its own right). You can also solve it by only joining tiles wherever you can maintain the original gradient (significantly less efficient but by definition still more efficient than having a vertex at each tile corner). I don't recall how Nocte solved it, I'll have to look later.
In any case, this is a nice way to shave off some geometry that doesn't really cost anything after the initial tessellation. Another complementary approach is occlusion culing. If you can guarantee some geometry will not be visible to the player, you don't need to draw that chunk, which improves GPU. If you wanted, you could even avoid loading the geometry onto the GPU until it became visible. Since most of the below-ground geometry will never be seen by the player, this is an attractive option. Furthermore, the latest snapshot actually has an implementation of software occlusion culling, so this wouldn't be all that difficult to add.
Still, none of this addresses the fact that we can't reduce the detail of visible geometry without it being obvious if we're using polygons. My research has been with finding a way to use shaders and raymarching to at least pretend that the blockiness is still present. I've had quite a bit of success on the front with natural heightmaps. Thanks to the way bilinear interpolation works, I can algebraicly solve for the intersection with the smooth height surface and the camera ray, and from there terrace and then blockify the results to minimize the amount of fishing around the ray marcher has to do. At this point it's cheap enough that it really doesn't impact the fps, but there's still more to add before it's complete.
Trees, buildings, and other volumetric structures can't be drawn with this system, and I don't have a great solution for that. My hope is that by saving enough geometry with the above systems, we'll have enough left in the budget to do some regular polygon rendering way out in the distance. There are also a number of techniques for generalizing a voxel model into a simpler voxel model, but in this case, the texture really needs to be preserved, and I don't have a great way of efficiently mapping a procedural texture to an arbitrary mesh. Ways do exist, but they are complicated and expensive. A different thing we can try is a forced depth of field, like what they did in Far Cry 3 to obscure their low LOD environment. Once the blockyness becomes undetectable, tons of options for reducing LOD with smooth meshes become possible.
=================== Meta Layers ======================
One of the earlier discussions was about fluid dynamics and how that might work. Back in the /indev days of Minecraft, water was finite and could be moved around or removed from an area by letting it dry out. The surrounding sea worked as infinite source blocks, and any water block connecting to the sea became an infinite source as well. It's hard to convey just how great that was if you haven't tried it. I wholeheartedly recommend dialing back your launcher to /indev and giving a try. Instead of annoying streams running down tunnels, you could completely flood a cave if you broke through the wrong wall, and a lot of the caves were already flooded, so to explore you'd have to dive in, and pop up in other chambers. We'd use sand held up with torches to act as floodgates since doors and signs didn't exist yet. It was seriously the best thing ever.
Once maps became unbounded, this was removed, since there was no convenient way to work out how much water you were interracting with at any given moment. But what if there was? Just like how we could lower the detail with light propagation and geometry, what if we could do the same thing with lakes and oceans? Imagine if instead of trying to count all the individual water blocks in a lake, you have a single object which says "this is a lake. It contains X gallons of water and is fully contained in region A, excluding regions B and C. Any water block there is part of the lake". Now you know how much water is in the lake, and what that will do to the water level if you channel part of that into a pond. This is also data you can keep in memory and interract with even if the chunks are unloaded from memory. If you wanted to get fancy with the fluid dynamics, now that you know the region, you can further subdivide the area to calculate flow through bottlenecks, and also calculate what will happen even if the water source is kilometers away without loading all the chunks in between (assuming we aren't worrying about water interracting with anything along the way, like redstone, but then you could model redstone circuits the same way).
For Futurecraft's engine, I have layers for any sort of thing that can't reasonably be accomodated on a per-chunk basis or doesn't fit the 1:1 block scale (like the lighting). These layers can be loaded and streamed from the server independently from the chunk layer, so long as there is no direct interraction while no players are present. Tying everything to the blocks and chunks was the biggest mistake Minecraft makes. You have a physical meta layer, but you have to first find out what each block is before you can interpret its meta. Ideally, any blocks that interract in a system, like redstone or buildcraft pipes or whatnot, should at least partially live in a data model where every component knows precisely how it relates to the rest of the system, as well as any additional system info (like current running through a stretch of wire). Otherwise you're just spamming the chunk cache just to work out basic relationships, or wasting more expensive tile entities simply to store a couple extra piecs of data.
That's a basic overview of about three years worth of research and forum discussions on the subject of cubic chunks and scaling Minecraft. There have been a lot of interesting and clever proposals which I wasn't able to get into, but the above ones are my favorite, and are what I'm actively attempting to implement for my own project.
If anyone's interested in any of these ideas, I can go into greater detail about how it would work and the challenges involved. Minecrak, you were involved in Robinton's CC thread a lot more than I was, if you can recall some of the other proposals that were made over there, let me know, my memory's a little fuzzy.
TL;DR NASA's impossible propulsion drive may be explained with a few small adjustments to the standard model.
In response to the queries about the fighters being entities, there's no two ways about it really. I mean who wouldn't love appearing as a lightly defensed frigate, but when in danger, a mother ship for a squadron of fighters. As for the size of the fighter, maybe a three block wingspan. At least small enough that you can have two fighters coming out of the hold of a ship at once. That will be a thing right? I mean the bigger ships carrying smaller ones. It will also add power in numbers even if you can't all afford bigger ships, and allow the ship to be better, I mean 5 people funding a ship and some fighters is better than one guy with his ship. But I am sure however it happens, it'll be great.
The point of entity-based vehicles is indeed that you can make them smaller whilst still having logically functioning parts, but three blocks from tip to tip is really, really tiny for something you are supposed to fit in. We'll probably end up keeping the scale from the Flans mod.
Btw, I'm making an effort to allocate more time in my week to work on FC. Can't guarantee it will last, but if I do anything interesting I'll talk about it here. Right now, it's looking like it will be relating to either the far draw system, which I have been neglecting for far too long, or some collaboration work with our friend MMM_MasterM of the Meta Worlds mod. Specifically, structuring it to weather the upcoming block state updates, and whatever is going to happen with dimension storage. From what I've read, it seems like it will only be a good thing. I'm really hoping that's the case, because it is an important step towards having a proper modding API.
I've also started getting back into the MC reverse-engineering game (I've been doing my framework stuff in C++ for a long time), mostly focusing on the 1.7.2 code (MCP is still catching up after the massive class restructuring, and more fields than not are still unlabeled, unless there is a new naming table somewhere I don't know about. I should probably look into that, actually.) Roughness of the translation aside, I am loving the new structure. It is much more organized and consistent, though the sheer amount of practical inheritance makes it difficult to anticipate all the various behaviors object variants might have. I'm pretty sure Eclipse has a way of outlining this in a meaningful way but I am still relearning the IDE.
Speaking of Eclipse, if you haven't updated to Kepler, holy crap is it ever awesome. I was impressed by the live editing in older versions, but now it is damn near instantaneous. It's incredibly convenient for messing around with the drawing routines. Can't wait to see what other goodies this bad boy has.
My logs are scattered all over two forums. Not even I know where all of it is anymore, and I know for a fact that some of it has disappeared. But the last few months have been really unproductive for me due to my IRL responsibilities becoming ever more demanding. We're not throwing in the towel, but don't expect a whirlwind of news anytime soon.
*edit*
From me, anyway. The FC content pack for Flan's on the other hand is shaping into something really cool (skip to the end for a recent build).
Erm guys. Not quite Future craft, but still.
Pair it with the archimedes ship mod and you have the basic idea of futurecraft
I wouldn't really put this in the same category. It still uses all the vanilla core engine mechanics, whereas Futurecraft will have lots of underlying changes in addition to movable chunks. No universal position or frame of reference, for one thing.
I do like the controls, though. They are advanced enough to get the job done in a self-contained system without a ton of extra crap interfacing it physically to other blocks, so you can do more advanced stuff than basic wire signals cleanly. My biggest complaint with command blocks has been how little they individually accomplish for all real-estate and processing time they eat up. That sort of thing makes much more sense as a fully-fledged scripting layer running out in the aether rather than tied to specific blocks.
It'll be a hybrid of ship-level HP and block-level procedural damage. Blocks are destroyed locally, but the point at which they start being destroyed and how much resistance they put up is based on overall or regional HP. More damage = easier to destroy. There is also a difference between armor-piercing, explosive, and energy damage in terms of thresholds and rates for HP and block damage. System failures and breaches will then be tied to block destruction, so layout still matters.
What we are trying to avoid is the idea of doubling your hull doubles your staying power. This will lead to what we call Borg Cube syndrome and will kill any semblance of balance. Plus, this way gives us more traditional RPG control of stats due to materials and equipment and will permit more artistic freedom when designing vessels.
We've also discussed adding energy shields as a counter to armor, so that bulkier Battlestar Galactica style ships which rely on armor placement can still exist logically in the same universe as fancier, more fragile looking, for lack of a better word Space Elf ships. The idea is that the two are mutually exclusive in their efficacy, and using each will require different strategies and tactics.
The plan is to have something between a community cloud network and dedicated servers. The big thing we're aiming for is separate terrain databases for inactive terrain download. That will lighten the bandwidth and memory requirements for an active server instance, and hopefully this equates to handling more worlds with less hardware. There's been an ongoing debate as to what server ownership will entail. Due to the nature of this project, we don't have the resources to host it all ourselves, so there will need to be perks to donating to the public cloud, but at the same time we can't allow item spawning or other normal admin powers without breaking the game balance.
Right now, it's still a moot point without being able to predict the size of the community or how large the galaxy needs to realistically be.
From a technical standpoint, server instances are things like planets, space stations, and general outer space. Within a solar system, all travel will be seamless, with clients collaborating with multiple servers simultaneously. Theoretically, it should even be possible to have multiple servers managing a single planet if there is a need for it.
Solar systems will be connected to adjacent systems via stargate, which will let us control the topology of the network and keep more capable servers towards the core of the galaxy.
I have no idea if its good i just used Google translator. To the point i was more wondering if the spaceship entity movement was done in the over-world and not in space. I was just looking back to the size matters not video.
Sorry, the following will probably be difficult to follow, especially if you are using a machine translation. I'm not great at explaining things simply.
"Size matters not" was only about a week of work to see if the technique was possible. At the time, everything was done in the overworld, and you can see that in other videos on my channel. However, there is no particular need for it, other than convenience. To my knowledge, three other mods have also independently implemented the same method, only with separated chunks: Ships and Boats, Physics Craft, and most recently, Metaworlds.
Ultimately, the plan is to detach the coordinate system from any single world or model, and allow entities and worlds to freely interact without a clear distinction as to which is the master world. This measure is necessary to accomplish the goals of this mod, and any half-measure hack is just going to cause trouble in the long run. Some time ago, I tried to do it with a fundamental rewrite of the World class, but it proved too problematic; literally every single entity and game mechanic assumed there was a single master world and that everything is aligned to that space. Any content changed between updates would have to be completely rewritten to accept the new model, and it would be a janitorial nightmare.
Instead of pushing forward with that, we fell back to the proxy entity approach used in the size matters not demo (though you never see collisions in that particular video), in which all independent world objects interact as entities rather than chunks. You can think of this paradigm like Relativity. Every entity believes their own frame of reference to be the master, but every frame of reference is equally correct. The proxy entities then act as a stand-in or adapter between two frames of reference.
Normally, there would be a heavy performance penalty by depending too much on the general entity pipeline for chunk interractions, which we counter in two ways: The first is to create highly optimized collision libraries for the most expensive chunk-on-chunk intersection tests, which take advantage of the fact that all blocks in a chunk are uniform in size, orientation, and spacing, to strip a great deal of the redundancy in box-vs-box collisions. For the second counter, we overload the collision management so that any object being tested for collision will know the identity of the other object, and can access that object's data. I believe Zeppelin mod was the first to use this, and this was the reason they couldn't become Forge compatible. I haven't tried decompiling Metaworlds, yet, and I have no idea how it pulls off the feat.
Additionally, every entity needs to be made compatible with 6 degrees of freedom collisions, as well as multiple pulls of gravity. For some entities, this isn't too bad. For others, it is really awful. I'll keep looking for an elegant solution, but I suspect there isn't one. Minecraft does not have anything you could really describe as a physics engine; everything that looks like one is a hack to achieve the same result. There is no mass, no momentum. Hell, there's not even a unified gravitational constant (for instance, a player will fall faster than a block of sand). We need some physics for space and floating debris, but mostly we are going to rely on lore-friendly mechanisms to sweep inconvenient discrepancies under the rug.
Also if the ship building and core stuff was released i think it would get a lot of pressure off the team if that's done yet.
It's not a bad idea. We had planned on doing all the engine stuff before starting on content, but since my ability to meet deadlines keeps becoming less and less reliable, that may not be realistic. Our biggest problem in the past has been a lack of organization and well-defined objectives, which makes it difficult for other programmers to contribute. As the lead developer, that is mostly my fault. Right now, our regular forum members are working to reorganize the project and redefine our goals. Hopefully this will help us get back on track. I really am sorry for all the delays. I would do more if I could.
Come on guys this is ridiculous at this rate the last time I asked your engine was not even started. What I am saying is that it would be great if he could get on ships in space, even if they do not battle and barely moving. I know me and my freinds would do that for hours.
Here's the same thing in Arabic for frost go right to left .
هيا الرجال هذا أمر مثير للسخرية في هذا المعدل آخر مرة سألت المحرك الخاص لم يكن حتى بدأت. ما أقوله هو أنه سيكون أمرا رائعا إذا كان يمكن الحصول على السفن في الفضاء حتى لو كانوا لا معركة وبالكاد تتحرك. أنا أعرف لي وفريندز بلدي سيفعل ذلك لساعات.
Did I do it right? The forum is doing something crazy with the direction markers, so it's hard to tell without being able to read it.
On-topic:
It's been mentioned before, but if you are interested in flying ships you can play with right now, I recommend checking out the metawords mod http://www.minecraft...nd-forge/. The technique being used in that mod is very closely related to what Futurecraft uses, and it is Forge compatible to boot.
1
@4HeadTiger: What I meant with my line about the 1:1 interaction was that only the boundary-changing edges of a body of water need to actively query the chunk. Think about the way water works now. When a block is at the edge of a flow going down a hill, it will expand based on the surrounding blocks. If the water block's neighbor rules have already been satisfied, the block is stable and will not change again unless a foreign block interacts with it. This can be applied to the dynamic model as well. Worse case, you'll end up with maybe a thousand active water block spanning a handful of chunks which are actively deciding whether to spread or trigger a block update. Everything behind them are stable and can be handled with the abstract model. If that's the numbers we end up dealing with, that's manageable without going to extra lengths to avoid it.
A special case, though, is waterfalls and water moving down slopes. Over a window of time, their flow rate should remain constant if their source and sink do, but depending on the flow rate, it may not look static every tick. I'm not completely sure what to do about them, but I feel like they shouldn't require direct chunk access if no one is in the area.
Edit:
With runoff, I'm still concerned about water collecting in unwanted places. In caves, for instance, since the things weren't cut out with water, they're not going to have a proper drainage route, and if a player builds in a dry valley, that valley is going to fill up over time unless we can drain water through dirt and rock. I'm just not sure it's safe to let the server blindly distribute water without some idea of where is and isn't a good place for it to end up.
1
In the meantime, hydrodynamics!
This part concerns me a bit. I'm not sure how this is intended to mesh with your concept of chunk-level statistics, but it sounds like you would still be handling certain liquid interactions at a 1:1 scale. I don't doubt that this is necessary in order for a stream of water to travel over new ground, but might we be able to make a distinction between water blocks actively altering the boundaries of a body of water and those which are merely flowing internally?
Edit: wait, I think that is sort of what you are suggesting; separating the distinct blocks a player will see move around from the macro-scale water mechanisms.
Additionally, I would model the water collision layer a bit more simply. You have your solid blocks, your air blocks, blocks with will trigger important chunk updates (e.g. redstone), and blocks with unimportant chunk updates (e.g. grass). If a block needs an important chunk update, it's probably not worth it to further guess at its behavior without simply loading the chunk up and interacting with it.
I can get behind this idea, but I'm not sure the chunk layer is the level I would store it. If it were me, I would treat a contiguous body of water as a single abstract object (with special handling for oceans. I would also try to subdivide that body of water into convex hulls and graph the internal flow rate between them by using the surface area of the hull borders. That won't solve the complications of territory altering water flow, but it should scale well once a body of water has stabilized.
Otherwise, you'll be running into the issue of two isolated streams of water occupying the same chunk when we zoom back out.
As long as the water channels are stable, this sounds about right. That's what you meant, right?
If we are guaranteeing that a fixed amount of water exists in the hydrosphere, wouldn't it just be more busy work to simulate precipitation when we know what the results ought to be? I mean, unless you plan on simulating droughts. My concern is that simulating things like runoff could be expensive to handle procedurally, and if it results in water appearing in places not originally intended to house water, it might end up causing trouble for players who might not be happy to discover their mine has transformed into an underground swimming pool over the course of a month.
In any case, I'm of the strong opinion that water propagation should not be handled without direct chunk interaction if it is unclear what the results are going to be. That's just asking for headaches. If stable flows can be abstracted to a less expensive simulation, I don't think the remaining roaming flows will be prohibitively hard hard on the budget, especially if we can tell beforehand whether a chunk update is going to be invoked.
There actually is one bizarre aspect of terrain loading, and that's decorations. Once you generate your volumetric model and convert it to biomes, you have to decorate it with ores and other block variants, followed by vegetation and macro structures like villages, ravines, and temples. At 5km, these things aren't small enough to simply ignore, at least I don't think so, and vegetation is by far the worst. At present, I don't have a good way of mapping trees, but it would be very obvious if they suddenly decided to show up as you got closer.
With the hydrosphere, bodies of water will also need to be added after the initial terrain is analyzed.
Nah, this is just haggling at this point. But it doesn't hurt to keep this stuff in mind so we don't actively sabotage it getting added down the road.
I agree with this except for the lighting model. Considering that this has always been the stumbling block in every cubic chunks attempt, if we don't come up with a practical solution this mod will end up dead in the water regardless of success on any other front.
I would also recommend holding off messing with occlusion culling until 1.8, since it will be part of the pipeline by that point.
2
Oh, you're such a tease.
Alright I'm sold, hydrodynamics is now in the definitely column.
In seriousness, though, another important element of rivers you may not have considered is how terrain is generated. The world is generated from layered Perlin noise, obviously, but unless there are an infinite number of octaves, the pattern begins looking uniform when you pull far enough back. The Island continents can't really get much bigger than they are now unless we add another octave or two to the noise function (not a bad idea, really, since the features all start to look miniature with an upped draw distance).
But even if we got those bigger, more realistic terrain features, we still can't expect to scale up to IRL sized features and phsyics without getting unwieldy for the player. For instance, the map of Skyrim is only 4 miles from one end to the other, but it feels expansive because you are on foot and there's lots of detours along the way. With that in mind, we need to have a way to handle fluid dynamics that still works on a much smaller scale while keeping things in equilibrium.
My proposal is scaling the virtual capacity of a body of water disproportionately to its physical volume. For instance, a small pool with a hundred blocks of water is treated as containing 100 blocks of water. However, a lake a hundred meters across may be handled like it contains 10 or 100 times as much water as it actually does. Oceans would effectively be infinite.
Combine that with rain and tributaries from mountain runoff (a reasonable feature with a larger terrain scale) and it would be possible to keep the water levels stable over a period of time. Plus, during the initial terrain generation it should be possible to only place lakes where equilibrium is achievable. If we can then rely on the assumption that equilibrium is the natural state of rivers and lakes without any direct player intervention, that should make life a little bit easier for the macro-scale simulation.
Then there's Futurecraft, which is the whole reason I'm investing research into cubic chunks. In Futurecraft, you live on cubic planets which are ~5km to a side (the whole surface has to be pre-generated, and that's about as large as can be reasonably handled), If we came up with some way to simulate a fully functional (or at least functional-looking) hydrosphere down at that scale, that would just be the best thing ever.
Whether we actually implement it or not, I think it is feature definitely worth humoring, and trying to accommodate it as we go about reworking other aspects definitely going in, like terrain generation.
2
I'll collect some notes and visual aids this weekend. It's been a while so I need to do a refresher anyway.
The intent isn't so much to support more realistic lighting (though I doubt anyone would complain if that was the end result). Rather, it's to get something, or anything really, to act as a surrogate for the "light shafts travel indefinitely and illuminate everything along the way" effect without resorting to querying individual blocks. What that effect is supposed to do is act as a surrogate for the idea that 1) light bounces off a surface and lights up the surrounding area (global illumination), and 2) scattered light from the sky and ground prevents anything not directly lit by the sun from getting particularly dark (macro-scale ambient occlusion).
If we ignored 2), we could just do the regular light diffusion at the point where the sunlight hits a surface. That would solve the scaling issue, but anywhere there was a sheer vertical drop, like cliff faces and mine shafts, would be left in complete darkness. On the moon, if the sun was perfectly overhead, this is exactly what would happen. But we live where there is a sky, so 2) needs to be taken into account in some way.
The reason for spherical harmonics isn't to represent direct sunlight, in fact you would explicitly ignore direct sunlight since it acts like a beam and doesn't diffuse. The thing it's representing is that directional light from the sky and the ground. The reason I say it's directional, even though it's in every direction, is due to the cases where it's partially occluded by a tree or a cave. The further underneath it you get, the darker things become, not because you are further away from the direct sunlight, but because you are less exposed to those ambient lights.
If we had a different way to approximate macro-scale ambient occlusion, we could use that instead. But say you're working out the lighting for a giant cave, maybe it's fully enclosed, or maybe we're near the mouth, it's tough to tell. In a case like that, it may end up being just as much effort to do "shortcuts" as it would be to just bite the bullet and record some real numbers.
I should probably also mention that I expected the light propagation to be either fully server-side or at least server-assisted. The results would still get transformed back into discrete 4-bit light levels for rendering and interaction (but this only matters within the region actively experiencing ticks, we could do whatever we wanted in scenery beyond that).
I have my doubts about that. This is the the defining point where everyone's attempts to-date have broken down, and is the thing spooking Mojang away from doing it themselves. No amount of adaptation to 1:1 light propagation will ever permit it to scale, so one way or another we'll have to go off the deep end if we want this. Not necessarily spherical harmonics, but the formal alternatives aren't especially entry-level, either.
VBOs have actually been supported for a really long time. If the GPU supports it (and realistically that's all of them), geometry is referenced from the stored buffer. If not, the polygon is given the draw command then and there. Either way, everything is getting compiled to display lists, which are immutable collections of optimized GL commands that get executed in bulk. Display lists are now depreciated, because they clash with the design philosophy of the programmable shader pipeline, but functionally they still work the way they always have and there's no advantage performance-wise to removing them.
What is interesting is there was a MogMiner tweet not too long ago saying they were going to remove all the fixed-function GL commands for 1.9. I'm not sure if that extends to the display lists, but if it does, that's going to create a major architectural rewrite of the graphics pipeline. I kind of hope they do, since it could be fantastically powerful boon for shader modding, but it could also wreak havoc on the programmatic way entities are rendered. Or maybe they'll support both methods for a while to ease the transition. We'll just have to wait and see what happens.
Only loading the faces visible from the camera's direction has been proposed before, but it's almost more trouble than it's worth. With backface culling, polygons facing the wrong way (as determined by a right-hand rule in the vertex ordering) are completely eliminated before incurring a pixel fill cost, though the vertices still have to be transformed into screen coordinates before this can be determined. Assuming you are not swamping the command pipe, you could split up each chunk into 6 layers, one for each face, and draw 3 of them in most occasions, but even with an upper bound of 50% savings (which is unlikely), thanks to the law of cubes that would only let us extend our draw distance by another ~26% before we were back to where we started. The only way to put a real dent in the scaling factor is to get as close to O(n2) space complexity as possible. Anything less won't matter in the long run.
If you mean prerendering parallax backdrops, that can sort of work, but only if you are viewing it from roughly a single direction. If you circle too far around or fly overhead, the illusion breaks down.
Occlusion culling really needs its own post. I'll write something up regarding that this weekend.
You'd come to regret that when I become swamped with work and disappear for 6 months. Again.
Yeah, it's been done and it is pretty ugly. Some other blockworld engines have gotten away with 2x or even 4x scaling, but more than that and there's too much loss of information and it becomes distracting. If we are getting out to the point that we can use 1/16th the detail, a google earth style tiled texture projected onto a heightmapped mesh is a more traditional approach to game terrain and will look closer to the real thing than trying to retain blocks. It shouldn't be necessary, either, with the blockifying shader, but you were the one who brought it up.
8
Oh well, better do it all at once, then.
=========== Introduction =============
Obviously, the problem with supporting cubic chunks isn't one of storage or encoding, but the implication that two things can and need to interact on a scale that you can't possibly hope to simulate. No matter how efficient and tightly optimized your per-tick code is, the Law of Cubes is going to make you her *****.
So what can we do about it? Well, if you look at other voxel engines (of the volumetric modelling variety, not the Minecraft block worlds), you'll notice that to the man, every implementation features very small voxels, to the point that individual voxels become indistinguishable from one another (except for Ken Silverman's Voxlap engine, but that one's a freak of the voxel world). Most of the time, the developer will go to great effort to hide the discreteness of the voxel structure. Because that detail is obscured, it opens opportunities for all manner of corner-cutting. Oct-tree storage (subdiving the grid only where there is a meaningful improvement in fidelity), google-maps style streaming detail, acceleration structures (mechanisms mixed into the data model to tell the render where it can take shortcuts, like large empty spaces. The entire discipline is about making voxel data smaller, faster, and more intelligent.
...Except for block worlds. Around here, we care that everything is sharply defined and cubic, and we also care about 1:1 detail, which is why we can't leverage all those state of the art techniques that make voxels actually a viable, scalable product. But what if we could? In an effort to play by the rules of the vanilla Minecraft physics engine, we are artificially making things harder on ourselves than they need to be. If you managed to get a 2km draw distance, you would find that individual blocks create a awful looking Moire pattern, since only one polygon can occupy a single screen-space pixel at a time (without a supersampling AA which is expensive doesn't really agree with Minecraft). So why should we have to draw blocks that way? If you make a 31-wide platform a mile up in the air, Minecraft says the shadow it casts would be completely dark in the center. Even if it were feasible, it would look ridiculous, so why are we trying to do that? Same goes for water, why should we consider water on a block-by-block basis if it becomes impossible to handle fluid dynamics on a large scale?
========== Lighting ===========
Minecraft lighting simulates a poor man's global illumination-the idea that rather than hard light and shadow, light bouncing off surfaces lights up everything nearby, making a smoothish gradient. The other principle tenant of global illumination is that the strength of light isn't a function of distance, but the arc-length of the source. If you have a nearby lightsouce and a giant wall of light in the distance, but both take up the same amount of your field of view, they are contributing the same amount of light to your position. The practical thing this means for us is that in terms of light contribution (excluding direct sunlight), the farther away a light source is-the larger it has to be in order to have any influence on a particular surface, meaning you can be working with lower and lower resolution models and getting the same effect. It would be difficult to tell whether you are inside or outside in a cave the size of a baseball stadium, but scaled down to a hut, it's not so difficult.
I mentioned sunlight didn't fit this model, and that's because the sun is so large and far away that its arc length doesn't change appreciably no matter where you are, so assuming you are not trying to simulate the sun moving through the sky, all you need to know is the heightmap to see if anything directly above you is occluding the sun to know whether you are in shadow, no big deal. But I also said before that it would be ridiculous for a small platform a mile up to be casting any appreciable shadow. The reason for this is that direct sunlight is not the only source of light. You also have ambient lighting. To keep thngs simple, ambient light is light coming from no place in particular. In our case, it's coming mostly from the scattered light in the sky and reflected light on the ground. If you are out in the open, it's the same no matter where you go. If you are under a tree, the leaves are blocking the sky so your only ambient light is coming from the horizon and reflected off the ground. In other words, when not directly lit by the sun, the amount of light hitting a block is the arc-area facing the sky, lit environment, or nearby point lights.
If you fully simulate the contribution of light from all these sources, and then recalculate again with those results, over and over until the new numbers stop changing appreciably, then you've just done global illumination, and the results will look hella good. On the other hand, it's also hella expensive, to the point that only recently has anyone even attempted doing it in realtime with the beefiest of GPUs. Additionally in the close range with block lights, and with direct sunlight, the current system isn't a terribly bad imitation. It's only those intermediate distances, where you're in the air, not next to any block light, but you're also not directly lit by sunlight, where the immitation starts breaking down. The lighting algorithm also spends a disproportionate amount of time lighting up these places that don't actually contribute anything. So why not scrap the idea that sunlit spaces need to radiate?
This next part is hard to explain, so I'm just going to give a rough outline for now. There is a complex system of equations in the world of mathematics called spherical harmonics, which are handy for representing directional lighting through a point in space. What it is is a set of equations which describe any arbitrary combination of directional magnitudes in 3D, translated to a set of coefficients. It's very similar to how you can recreate any sound by combining enough sine waves at the right frequency and amplitude. To apply lossy compression on a sound, you try to remove all the sine waves that don't significantly affect the result. Jpeg compression works more or less the same way, first converting color gradients to waves. The same applies to spherical harmonics. By adding or removing coefficients, you can increase or decrease the accuracy of the representation. What this means in practice is that we can represent light rays passing through a point from an infinite number of directions with a relative handful of parameters. Just don't go trying to simulate a laser pointer.
So now you can imagine a world filled with these things, one in the middle of each air block, each telling their neighbors how much light that vector is contributing. That takes care of getting light to travel in all directions, but it's still really expensive per point, and we're processing even more points than the old way. That's where the scaling rule comes into play. Light sources farther away need to be larger to contribute any meaningful light, so you can handle them at a lower resolution. As far as physical placement, this means larger open areas can be represented with fewer spheres. Out under the sky, they would be huge and paint with massive strokes, which is what you would expect from ambient light. We'd still use direct sunlight and point lighting, as well as scattering in the places where sunlight actually strikes (not simply passing through), but the big empty spaces are now vacuums that light simply passes through.
There are a number of considerations with this, on top of the math being fairly gnarly, like what arrangement and density of spheres gives us the best bang for our buck, and will this simulation work if there is a large difference in brightness between the sun and block lighting? Additionally, though it would be less noticeable, we'd need to simulate light propagation independently for block lights. Individual points of light die off pretty quickly, but in a big room full of point lights, the numerous little contributions add up and even-out the light levels (this is floating point math, so tiny contributions are recorded just fine so long as all the contributions are in that same range, which is what we want). With the right tweaking, and especially using the assumption that the sun rays are stationary, there's no reason this shouldn't be feasible, at least server-side. Distant changes are also less noticeable, so there is no reason long-distance light propagation has to be done all at once, either.
=========== Block Rendering ============
I won't go into great detail here, because this is part of my active research and there's too much to cover. Block geometry takes up a lot of space on a GPU. If memory serves, normal vertex stores world coordinates, texture coordinates, an additional texture coordinate for lighting, and a color for biome-sensitive blocks like grass, and a couple bytes of padding, multiplied by 4 vertices per face (and block faces never share vertices) adds up to 128 bytes per block face. That's a lot, and you run out fairly quickly, which is why the draw distance is so low. The worst part is, thanks to that law of cubes, every bit of range you increase the draw distance by increases the memory footprint geometrically, which no amount of culling or optimization is going to offset for long. What we need here, and what all those other voxel engines do, is implement level-of-detail.
The obvious problem behind LOD is that the block world environment wears its discrete details like a flag, and you can't simply reduce the geometry in the blocky parts without it being spectacularly obvious. If the texture doesn't change, you can join up adjacent block faces in the same plane to use fewer polygons. Nocte does this for Hexahedra, and it's been used by a couple other block world engines as well. Since lighting was being stored at each vertex, this can pose a problem when not every tile has its own vertices. One engine solved that by using a dense 3D texture of light values and married the light and geometry via shader (though a dense 3d lightmap is pretty heavy in its own right). You can also solve it by only joining tiles wherever you can maintain the original gradient (significantly less efficient but by definition still more efficient than having a vertex at each tile corner). I don't recall how Nocte solved it, I'll have to look later.
In any case, this is a nice way to shave off some geometry that doesn't really cost anything after the initial tessellation. Another complementary approach is occlusion culing. If you can guarantee some geometry will not be visible to the player, you don't need to draw that chunk, which improves GPU. If you wanted, you could even avoid loading the geometry onto the GPU until it became visible. Since most of the below-ground geometry will never be seen by the player, this is an attractive option. Furthermore, the latest snapshot actually has an implementation of software occlusion culling, so this wouldn't be all that difficult to add.
Still, none of this addresses the fact that we can't reduce the detail of visible geometry without it being obvious if we're using polygons. My research has been with finding a way to use shaders and raymarching to at least pretend that the blockiness is still present. I've had quite a bit of success on the front with natural heightmaps. Thanks to the way bilinear interpolation works, I can algebraicly solve for the intersection with the smooth height surface and the camera ray, and from there terrace and then blockify the results to minimize the amount of fishing around the ray marcher has to do. At this point it's cheap enough that it really doesn't impact the fps, but there's still more to add before it's complete.
Trees, buildings, and other volumetric structures can't be drawn with this system, and I don't have a great solution for that. My hope is that by saving enough geometry with the above systems, we'll have enough left in the budget to do some regular polygon rendering way out in the distance. There are also a number of techniques for generalizing a voxel model into a simpler voxel model, but in this case, the texture really needs to be preserved, and I don't have a great way of efficiently mapping a procedural texture to an arbitrary mesh. Ways do exist, but they are complicated and expensive. A different thing we can try is a forced depth of field, like what they did in Far Cry 3 to obscure their low LOD environment. Once the blockyness becomes undetectable, tons of options for reducing LOD with smooth meshes become possible.
=================== Meta Layers ======================
One of the earlier discussions was about fluid dynamics and how that might work. Back in the /indev days of Minecraft, water was finite and could be moved around or removed from an area by letting it dry out. The surrounding sea worked as infinite source blocks, and any water block connecting to the sea became an infinite source as well. It's hard to convey just how great that was if you haven't tried it. I wholeheartedly recommend dialing back your launcher to /indev and giving a try. Instead of annoying streams running down tunnels, you could completely flood a cave if you broke through the wrong wall, and a lot of the caves were already flooded, so to explore you'd have to dive in, and pop up in other chambers. We'd use sand held up with torches to act as floodgates since doors and signs didn't exist yet. It was seriously the best thing ever.
Once maps became unbounded, this was removed, since there was no convenient way to work out how much water you were interracting with at any given moment. But what if there was? Just like how we could lower the detail with light propagation and geometry, what if we could do the same thing with lakes and oceans? Imagine if instead of trying to count all the individual water blocks in a lake, you have a single object which says "this is a lake. It contains X gallons of water and is fully contained in region A, excluding regions B and C. Any water block there is part of the lake". Now you know how much water is in the lake, and what that will do to the water level if you channel part of that into a pond. This is also data you can keep in memory and interract with even if the chunks are unloaded from memory. If you wanted to get fancy with the fluid dynamics, now that you know the region, you can further subdivide the area to calculate flow through bottlenecks, and also calculate what will happen even if the water source is kilometers away without loading all the chunks in between (assuming we aren't worrying about water interracting with anything along the way, like redstone, but then you could model redstone circuits the same way).
For Futurecraft's engine, I have layers for any sort of thing that can't reasonably be accomodated on a per-chunk basis or doesn't fit the 1:1 block scale (like the lighting). These layers can be loaded and streamed from the server independently from the chunk layer, so long as there is no direct interraction while no players are present. Tying everything to the blocks and chunks was the biggest mistake Minecraft makes. You have a physical meta layer, but you have to first find out what each block is before you can interpret its meta. Ideally, any blocks that interract in a system, like redstone or buildcraft pipes or whatnot, should at least partially live in a data model where every component knows precisely how it relates to the rest of the system, as well as any additional system info (like current running through a stretch of wire). Otherwise you're just spamming the chunk cache just to work out basic relationships, or wasting more expensive tile entities simply to store a couple extra piecs of data.
================= Summary =============================
That's a basic overview of about three years worth of research and forum discussions on the subject of cubic chunks and scaling Minecraft. There have been a lot of interesting and clever proposals which I wasn't able to get into, but the above ones are my favorite, and are what I'm actively attempting to implement for my own project.
If anyone's interested in any of these ideas, I can go into greater detail about how it would work and the challenges involved. Minecrak, you were involved in Robinton's CC thread a lot more than I was, if you can recall some of the other proposals that were made over there, let me know, my memory's a little fuzzy.
TL;DR NASA's impossible propulsion drive may be explained with a few small adjustments to the standard model.
2
The point of entity-based vehicles is indeed that you can make them smaller whilst still having logically functioning parts, but three blocks from tip to tip is really, really tiny for something you are supposed to fit in. We'll probably end up keeping the scale from the Flans mod.
--------------------------------------------------------------
Btw, I'm making an effort to allocate more time in my week to work on FC. Can't guarantee it will last, but if I do anything interesting I'll talk about it here. Right now, it's looking like it will be relating to either the far draw system, which I have been neglecting for far too long, or some collaboration work with our friend MMM_MasterM of the Meta Worlds mod. Specifically, structuring it to weather the upcoming block state updates, and whatever is going to happen with dimension storage. From what I've read, it seems like it will only be a good thing. I'm really hoping that's the case, because it is an important step towards having a proper modding API.
I've also started getting back into the MC reverse-engineering game (I've been doing my framework stuff in C++ for a long time), mostly focusing on the 1.7.2 code (MCP is still catching up after the massive class restructuring, and more fields than not are still unlabeled, unless there is a new naming table somewhere I don't know about. I should probably look into that, actually.) Roughness of the translation aside, I am loving the new structure. It is much more organized and consistent, though the sheer amount of practical inheritance makes it difficult to anticipate all the various behaviors object variants might have. I'm pretty sure Eclipse has a way of outlining this in a meaningful way but I am still relearning the IDE.
Speaking of Eclipse, if you haven't updated to Kepler, holy crap is it ever awesome. I was impressed by the live editing in older versions, but now it is damn near instantaneous. It's incredibly convenient for messing around with the drawing routines. Can't wait to see what other goodies this bad boy has.
0
(First one is at the 5:14 mark. Stupid curse keeps stealing my timestamp.)
0
0
*edit*
From me, anyway. The FC content pack for Flan's on the other hand is shaping into something really cool (skip to the end for a recent build).
0
I wouldn't really put this in the same category. It still uses all the vanilla core engine mechanics, whereas Futurecraft will have lots of underlying changes in addition to movable chunks. No universal position or frame of reference, for one thing.
I do like the controls, though. They are advanced enough to get the job done in a self-contained system without a ton of extra crap interfacing it physically to other blocks, so you can do more advanced stuff than basic wire signals cleanly. My biggest complaint with command blocks has been how little they individually accomplish for all real-estate and processing time they eat up. That sort of thing makes much more sense as a fully-fledged scripting layer running out in the aether rather than tied to specific blocks.
It'll be a hybrid of ship-level HP and block-level procedural damage. Blocks are destroyed locally, but the point at which they start being destroyed and how much resistance they put up is based on overall or regional HP. More damage = easier to destroy. There is also a difference between armor-piercing, explosive, and energy damage in terms of thresholds and rates for HP and block damage. System failures and breaches will then be tied to block destruction, so layout still matters.
What we are trying to avoid is the idea of doubling your hull doubles your staying power. This will lead to what we call Borg Cube syndrome and will kill any semblance of balance. Plus, this way gives us more traditional RPG control of stats due to materials and equipment and will permit more artistic freedom when designing vessels.
We've also discussed adding energy shields as a counter to armor, so that bulkier Battlestar Galactica style ships which rely on armor placement can still exist logically in the same universe as fancier, more fragile looking, for lack of a better word Space Elf ships. The idea is that the two are mutually exclusive in their efficacy, and using each will require different strategies and tactics.
0
Right now, it's still a moot point without being able to predict the size of the community or how large the galaxy needs to realistically be.
From a technical standpoint, server instances are things like planets, space stations, and general outer space. Within a solar system, all travel will be seamless, with clients collaborating with multiple servers simultaneously. Theoretically, it should even be possible to have multiple servers managing a single planet if there is a need for it.
Solar systems will be connected to adjacent systems via stargate, which will let us control the topology of the network and keep more capable servers towards the core of the galaxy.
0
0
Sorry, the following will probably be difficult to follow, especially if you are using a machine translation. I'm not great at explaining things simply.
"Size matters not" was only about a week of work to see if the technique was possible. At the time, everything was done in the overworld, and you can see that in other videos on my channel. However, there is no particular need for it, other than convenience. To my knowledge, three other mods have also independently implemented the same method, only with separated chunks: Ships and Boats, Physics Craft, and most recently, Metaworlds.
Ultimately, the plan is to detach the coordinate system from any single world or model, and allow entities and worlds to freely interact without a clear distinction as to which is the master world. This measure is necessary to accomplish the goals of this mod, and any half-measure hack is just going to cause trouble in the long run. Some time ago, I tried to do it with a fundamental rewrite of the World class, but it proved too problematic; literally every single entity and game mechanic assumed there was a single master world and that everything is aligned to that space. Any content changed between updates would have to be completely rewritten to accept the new model, and it would be a janitorial nightmare.
Instead of pushing forward with that, we fell back to the proxy entity approach used in the size matters not demo (though you never see collisions in that particular video), in which all independent world objects interact as entities rather than chunks. You can think of this paradigm like Relativity. Every entity believes their own frame of reference to be the master, but every frame of reference is equally correct. The proxy entities then act as a stand-in or adapter between two frames of reference.
Normally, there would be a heavy performance penalty by depending too much on the general entity pipeline for chunk interractions, which we counter in two ways: The first is to create highly optimized collision libraries for the most expensive chunk-on-chunk intersection tests, which take advantage of the fact that all blocks in a chunk are uniform in size, orientation, and spacing, to strip a great deal of the redundancy in box-vs-box collisions. For the second counter, we overload the collision management so that any object being tested for collision will know the identity of the other object, and can access that object's data. I believe Zeppelin mod was the first to use this, and this was the reason they couldn't become Forge compatible. I haven't tried decompiling Metaworlds, yet, and I have no idea how it pulls off the feat.
Additionally, every entity needs to be made compatible with 6 degrees of freedom collisions, as well as multiple pulls of gravity. For some entities, this isn't too bad. For others, it is really awful. I'll keep looking for an elegant solution, but I suspect there isn't one. Minecraft does not have anything you could really describe as a physics engine; everything that looks like one is a hack to achieve the same result. There is no mass, no momentum. Hell, there's not even a unified gravitational constant (for instance, a player will fall faster than a block of sand). We need some physics for space and floating debris, but mostly we are going to rely on lore-friendly mechanisms to sweep inconvenient discrepancies under the rug.
It's not a bad idea. We had planned on doing all the engine stuff before starting on content, but since my ability to meet deadlines keeps becoming less and less reliable, that may not be realistic. Our biggest problem in the past has been a lack of organization and well-defined objectives, which makes it difficult for other programmers to contribute. As the lead developer, that is mostly my fault. Right now, our regular forum members are working to reorganize the project and redefine our goals. Hopefully this will help us get back on track. I really am sorry for all the delays. I would do more if I could.
3
Well that simply won't do...
I like to think of it like the Tortoise and the Hare fable. In that all the spectators immediately regretted their decision to watch.
1
Did I do it right? The forum is doing something crazy with the direction markers, so it's hard to tell without being able to read it.
On-topic:
It's been mentioned before, but if you are interested in flying ships you can play with right now, I recommend checking out the metawords mod http://www.minecraft...nd-forge/. The technique being used in that mod is very closely related to what Futurecraft uses, and it is Forge compatible to boot.