(1.8 update = "hey let's C&P slime blocks from another mod! ")
A little bit off-topic, but they didn't C&P slime blocks from any mod, the slime block functionality of them sticking blocks to pistons was coded by the Zipkrowd guys specifically for it to be added into the game by Dinnerbone, and then he made the slime block piston launching himself, at least know what you are talking about before you bash Mojang.
Anyways, on-topic now, I took a look at Minecraft's lighting engine once and just... It was not pretty, so the fact that you managed to fix and clean up 75% of it in the time frame that you did is still really impressive, Cuchaz.
Rollback Post to RevisionRollBack
The problem with the truth, is that it never lies.
A little bit off-topic, but they didn't C&P slime blocks from any mod, the slime block functionality of them sticking blocks to pistons was coded by the Zipkrowd guys specifically for it to be added into the game by Dinnerbone, and then he made the slime block piston launching himself, at least know what you are talking about before you bash Mojang.
To repeat your words; "At least know what you are talking about" before telling people likewise. As far as I know, ZipKrowd was using them on their server. Dinnerbone saw them and asked the ZipKrowd dudes to make it fit into the vanilla. Cue the Ctrl, C and V keys.
In fact, even if he had requested entirely new content that had never been seen before, the end result would still be functionally identical in this argument. As for an entity launching from a piston, and I don't mean to imply anything negative about DB; I don't even use Java and I can do that (though I do use C++, R, and Python a little bit/daily). The idea is still nice, but the effort and skill required to implement it is non-existent.
As for "bashing"; it wasn't my intention in my second comment, and I can only apologize if that's how it came across. Mojang lacks productivity. This is indisputable, and "laziness" is one of the kindest ways to explain it. They also seem to have an attitude of "it all pretty much works without crashing, so let's just leave it and do some other feature now". Beyond this, my point was really that the dev team is conservative, which is understandable, as they are faced with numerous constraints (can't break worlds, have to keep a large community happy, etc).
As for not having a "coding wizard": that doesn't mean that the Mojangers aren't skilled programmers. Coding wizards are like the Mohammed Ali's and Joe Louis's, but with less spotlight, a bit less muscle, and a lot less exercise.
Back on topic, I agree that the 75% overhaul is quite remarkable. I had a look at the source on the bitbucket, and it looks nice and tidy, as well as stable and efficient
One question; ProcessBatch(...), is this just a kind of asynchronous loop? Just, I notice that in many cases, you add failed lighting calculations back to the queue to be processed again later, yet with a regular for or while loop... well... a persistent issue would create a massive explosion.
EDIT: I just had a look at your site. Congratulations on your PhD, Dr. Cuchaz ^^
Yeah, I have some negative feelings towards Mojang atm (1.8 update = "hey let's C&P slime blocks from another mod! :)").
-snip-
Yeah, there is certainly reason for Mojang to be conservative. There is little to be gained by them by taking big risks. I can't say I blame them. That's fine by me. It gives us modders some more fun things to do.
Back on topic, I agree that the 75% overhaul is quite remarkable. I had a look at the source on the bitbucket, and it looks nice and tidy, as well as stable and efficient One question; ProcessBatch(...), is this just a kind of asynchronous loop? Just, I notice that in many cases, you add failed lighting calculations back to the queue to be processed again later, yet with a regular for or while loop... well... a persistent issue would create a massive explosion.
No flame wars in my threads. Be civil, or take the discussion elsewhere. I'm not accusing anyone of un-civility yet, but don't let it escalate.
Anyway, yes, I changed the lighting system to be more asynchronous. In places it already was actually, but now much of the lighting calculations are consolidated into one package of classes. The queues in the lighting system are processed once per tick, so it won't be possible for the queues to explode within a single tick. It could be that the queues accumulated failed tasks over many ticks, but I think that won't have a major impact on performance. The thing that adds tasks to the queue always related to cubes. As long as the cubes are still loaded, the lighting task is valid and we should try to processes it. However, when the cubes eventually get unloaded, those lingering lighting tasks will try to get processed, but then short circuit and get dumped from the queues. As long as unused cubes get cleaned up (which they do), the lighting system should be self cleaning as well.
Anyway, yes, I changed the lighting system to be more asynchronous. In places it already was actually, but now much of the lighting calculations are consolidated into one package of classes. The queues in the lighting system are processed once per tick, so it won't be possible for the queues to explode within a single tick. It could be that the queues accumulated failed tasks over many ticks, but I think that won't have a major impact on performance. The thing that adds tasks to the queue always related to cubes. As long as the cubes are still loaded, the lighting task is valid and we should try to processes it. However, when the cubes eventually get unloaded, those lingering lighting tasks will try to get processed, but then short circuit and get dumped from the queues. As long as unused cubes get cleaned up (which they do), the lighting system should be self cleaning as well.
So long as the cube is in a state where it can be rendered, lighting will work properly, otherwise it gets discarded by default.That is just... man I wish I could pull stuff like that off without making everything go poop.
I'm really interested in how this will turn out in the end. Judging from your video, the vertical range for loading chunks was waaay too short. But I'm sure you'll find a resource-efficient and fast way to overcome such problems .
Keep up the amazing work!
It's definitely way too short. I kept it short because it makes errors easier to see and helps with debugging issues. It also makes the effects more apparent in the video. When I get the system working for realsies, you hopefully won't see empty spaces in the world at all. =)
Also, 75%??? Dear lord... that's in a released game... how did they not fix that before release!
Thanks! I just graduated a couple months ago.
There wasn't really anything seriously wrong with the old lighting system. It was a little messy and dispersed in terms of code design, but it got the job done. But when I changed the rules by handling terrain data differently, I broke some assumptions the old lighting system relied on. That's why I had to rewrite most of it. I'm making big changes to the game, which means other things will have to change too.
There wasn't really anything seriously wrong with the old lighting system. It was a little messy and dispersed in terms of code design, but it got the job done. But when I changed the rules by handling terrain data differently, I broke some assumptions the old lighting system relied on. That's why I had to rewrite most of it. I'm making big changes to the game, which means other things will have to change too.
Fair enough, though I am surprised the lighting system is as rigid as it is, they probably had reasons though.
Fair enough, though I am surprised the lighting system is as rigid as it is, they probably had reasons though.
A lot of dev teams have a kind of "If it works, then we can spend our time on other things" approach. Sure there were a couple of little glitchy things, but overhauling and rebuilding the entire lighting system is a complicated task, and since the current system worked without exploding and destroying everything: there was no reason to spend the time on it when that time could be better spent say, rearranging the entire data-pipeline within the game.
The only issue I see (I didn't spend long looking at it, so forgive me if I missed something) with your lighting system is that if the chunks above are not loaded, then the skylight calculator will think that the chunks below are open to the sky regardless of what was in the chunks above. This could, for example, break cave worlds and caverns, which is ~not good~. The conclusion on the CC thread was that a height-map system, storing the y coordinates of the highest known solid block at any x/y coordinate, was the best solution to this problem.
You could always simplify and use the solid/empty system to shortcut in some places;
if (isEmpty) { //create empty thing on heightmap }
else if (isSolid) { //create solid thing on heightmap }
else { //loop over every column individually (or array-erize the empty columns during generation?) };
But w/e. It's a bit of a change of pace, but I also think you are too modest in the thread title; CC will actually IMPROVE performance and REDUCE lag.
The only issue I see (I didn't spend long looking at it, so forgive me if I missed something) with your lighting system is that if the chunks above are not loaded, then the skylight calculator will think that the chunks below are open to the sky regardless of what was in the chunks above. This could, for example, break cave worlds and caverns, which is ~not good~. The conclusion on the CC thread was that a height-map system, storing the y coordinates of the highest known solid block at any x/y coordinate, was the best solution to this problem. You could always simplify and use the solid/empty system to shortcut in some places;
if (isEmpty) { //create empty thing on heightmap } else if (isSolid) { //create solid thing on heightmap } else { //loop over every column individually (or array-erize the empty columns during generation?) };
But w/e. It's a bit of a change of pace, but I also think you are too modest in the thread title; CC will actually IMPROVE performance and REDUCE lag.
This can be a problem, but only when the overhead chunks haven't been generated yet. And there's not much I can do about that without actually generating the chunks. If the overhead chunks are generated, but not loaded, then this is not a problem. I use a fancy version of a height map (look at the LightIndex classes) to solve this problem.
This can be a problem, but only when the overhead chunks haven't been generated yet. And there's not much I can do about that without actually generating the chunks. If the overhead chunks are generated, but not loaded, then this is not a problem. I use a fancy version of a height map (look at the LightIndex classes) to solve this problem.
Ah, I overlooked the LightIndex stuff. I should have looked at the source more
The generation thing is indeed a problem, and sadly it's as you say; the only solution is to generate the world higher :'(
EDIT: wow that is pretty fancy ^^
EDIT 2:
A problem that faced the old CC mod, and Bartek's attempt at a CC mod, was the vertical chunk loading speed. In fact, since the player can fall at about 80 blocks, or 5 chunks, per second; it is very easy for the player to fall into unloaded space no matter how efficient the system is, especially since you may need to load up to 10, or even 20 chunks per second to keep them falling without interruption if they are at the edge or corner of a chunk. How do you plan on dealing with this particular issue?
A problem that faced the old CC mod, and Bartek's attempt at a CC mod, was the vertical chunk loading speed. In fact, since the player can fall at about 80 blocks, or 5 chunks, per second; it is very easy for the player to fall into unloaded space no matter how efficient the system is, especially since you may need to load up to 10, or even 20 chunks per second to keep them falling without interruption if they are at the edge or corner of a chunk. How do you plan on dealing with this particular issue?
That's a tricky issue. The only way I can deal with it is generate some tall terrains and do some testing to see what works and what doesn't. I need to get a feel for the current loading speed first before I can decide what measures are needed to improve loading speed. And before I can do that, I need to generate tall worlds. And before that, I need to remove the height limits.
That's a tricky issue. The only way I can deal with it is generate some tall terrains and do some testing to see what works and what doesn't. I need to get a feel for the current loading speed first before I can decide what measures are needed to improve loading speed. And before I can do that, I need to generate tall worlds. And before that, I need to remove the height limits.
We're getting there. =)
The CC thread mentions that the falling would pause while the chunks load(the speed would likely be the same when it unpaused, wouldn't be to hard to do). Alternatively it would act like the horizontal chunk problems.
The CC thread mentions that the falling would pause while the chunks load(the speed would likely be the same when it unpaused, wouldn't be to hard to do). Alternatively it would act like the horizontal chunk problems.
The pausing is likely to be the best solution, combined with a slight cap to the max falling speed and so on.
The only thing to note is that where possible, it may be better to pause for longer and load many more chunks below the falling entity/player, since I think it would be much less annoying to get stopped 1 time for a 5 seconds than it would be to get stopped 5 times for 1 second each.
As for the horizontal chunk problems -> Technically I think the game does try to pause the entity until the chunk is loaded. The bigger problem there is that you usually only reach the edge of the loaded world when flying in creative mode, or when the chunks are corrupted in some way. Unfortunately, it is super easy to jump off of something tall in MC, and the last thing you want to do is poo on every player who wants to fall more than 99 blocks in a single jump. This means that there simply has to be a reliable system in place to deal with fast moving entities.
A possible solution though I don't know how practical that is, is to always prioritize the chunk directly below the player. That is, if the player suddenly starts falling, postpone loading of farther chunks and load only the chunks directly below him/her.
This will cause visual glitches, but will help keeping the player moving. We want as little to interfere directly with gameplay as possible, after all.
I like that idea. I'd personally rather fall at real time without freezing, even if I can't see everything around me, than pause so it can load other chunks around me.
Wait a minute... does it really mean we're close to huge height maps with possibly even higher FPS? And you've done it in few days? Please, tell me it's not a joke.
A possible solution though I don't know how practical that is, is to always prioritize the chunk directly below the player. That is, if the player suddenly starts falling, postpone loading of farther chunks and load only the chunks directly below him/her.
This will cause visual glitches, but will help keeping the player moving. We want as little to interfere directly with gameplay as possible, after all.
I raised the issue under the assumption that this would already be occurring. Even when the fall gets paused, it would only be to load additional chunks immediately below the player so that the fall can continue. The real problem is quite simply that;
You may need to load up to 10, or even 20 chunks per second to keep them falling without interruption
Sadly, we can't just say "meh, the player usually doesn't fall along the intersection line of 4 chunks, so we can just pretend it will never happen", because then the game might simply end up exploding whenever a player does it. In fact, all that would really need to happen for you to fall out of the world, even on a lightning fast system, would be for your disk to hang or delay a read operation for a few ms.
Of course, using a linear system, this isn't a problem, because the chances of you, for example, randomly accelerating to an x speed of 100 blocks per second, are extremely low. Takes an extra second to load up the chunk? Not a problem, because it takes an extra 10 seconds to ~reach~ the chunk.
However, the chances of falling off of a 200 block high structure and hitting the peak speed of 80 blocks per second, are actually quite high. This makes it much easier to fall out of the loaded world, and for this reason, any CC mod needs some reliable system in place to ensure that the game does not poo itself.
However, the chances of falling off of a 200 block high structure and hitting the peak speed of 80 blocks per second, are actually quite high. This makes it much easier to fall out of the loaded world, and for this reason, any CC mod needs some reliable system in place to ensure that the game does not poo itself.
Actually, according to the Minecraft Wiki, even falling 256 blocks you only reach 70 m/s, and the terminal velocity is ~78.4 m/s.
Still, taller worlds negates that fact, but 200 blocks will not get you to terminal velocity. Just wanted to mention that.
A little bit off-topic, but they didn't C&P slime blocks from any mod, the slime block functionality of them sticking blocks to pistons was coded by the Zipkrowd guys specifically for it to be added into the game by Dinnerbone, and then he made the slime block piston launching himself, at least know what you are talking about before you bash Mojang.
Anyways, on-topic now, I took a look at Minecraft's lighting engine once and just... It was not pretty, so the fact that you managed to fix and clean up 75% of it in the time frame that you did is still really impressive, Cuchaz.
In fact, even if he had requested entirely new content that had never been seen before, the end result would still be functionally identical in this argument. As for an entity launching from a piston, and I don't mean to imply anything negative about DB; I don't even use Java and I can do that (though I do use C++, R, and Python a little bit/daily). The idea is still nice, but the effort and skill required to implement it is non-existent.
As for "bashing"; it wasn't my intention in my second comment, and I can only apologize if that's how it came across. Mojang lacks productivity. This is indisputable, and "laziness" is one of the kindest ways to explain it. They also seem to have an attitude of "it all pretty much works without crashing, so let's just leave it and do some other feature now". Beyond this, my point was really that the dev team is conservative, which is understandable, as they are faced with numerous constraints (can't break worlds, have to keep a large community happy, etc).
As for not having a "coding wizard": that doesn't mean that the Mojangers aren't skilled programmers. Coding wizards are like the Mohammed Ali's and Joe Louis's, but with less spotlight, a bit less muscle, and a lot less exercise.
Back on topic, I agree that the 75% overhaul is quite remarkable. I had a look at the source on the bitbucket, and it looks nice and tidy, as well as stable and efficient
One question; ProcessBatch(...), is this just a kind of asynchronous loop? Just, I notice that in many cases, you add failed lighting calculations back to the queue to be processed again later, yet with a regular for or while loop... well... a persistent issue would create a massive explosion.
EDIT: I just had a look at your site. Congratulations on your PhD, Dr. Cuchaz ^^
I believe in the Invisible Pink Unicorn, bless her Invisible Pinkness.
-
View User Profile
-
View Posts
-
Send Message
Curse PremiumYeah, there is certainly reason for Mojang to be conservative. There is little to be gained by them by taking big risks. I can't say I blame them. That's fine by me. It gives us modders some more fun things to do.
No flame wars in my threads. Be civil, or take the discussion elsewhere. I'm not accusing anyone of un-civility yet, but don't let it escalate.
Anyway, yes, I changed the lighting system to be more asynchronous. In places it already was actually, but now much of the lighting calculations are consolidated into one package of classes. The queues in the lighting system are processed once per tick, so it won't be possible for the queues to explode within a single tick. It could be that the queues accumulated failed tasks over many ticks, but I think that won't have a major impact on performance. The thing that adds tasks to the queue always related to cubes. As long as the cubes are still loaded, the lighting task is valid and we should try to processes it. However, when the cubes eventually get unloaded, those lingering lighting tasks will try to get processed, but then short circuit and get dumped from the queues. As long as unused cubes get cleaned up (which they do), the lighting system should be self cleaning as well.
=D Thanks!
So long as the cube is in a state where it can be rendered, lighting will work properly, otherwise it gets discarded by default.That is just... man I wish I could pull stuff like that off without making everything go poop.
Btw, I love how you named these XD
I believe in the Invisible Pink Unicorn, bless her Invisible Pinkness.
-
View User Profile
-
View Posts
-
Send Message
Curse PremiumIt's definitely way too short. I kept it short because it makes errors easier to see and helps with debugging issues. It also makes the effects more apparent in the video. When I get the system working for realsies, you hopefully won't see empty spaces in the world at all. =)
Also, 75%??? Dear lord... that's in a released game... how did they not fix that before release!
-
View User Profile
-
View Posts
-
Send Message
Curse PremiumThanks! I just graduated a couple months ago.
There wasn't really anything seriously wrong with the old lighting system. It was a little messy and dispersed in terms of code design, but it got the job done. But when I changed the rules by handling terrain data differently, I broke some assumptions the old lighting system relied on. That's why I had to rewrite most of it. I'm making big changes to the game, which means other things will have to change too.
-
View User Profile
-
View Posts
-
Send Message
Curse PremiumNope.
Fair enough, though I am surprised the lighting system is as rigid as it is, they probably had reasons though.
A lot of dev teams have a kind of "If it works, then we can spend our time on other things" approach. Sure there were a couple of little glitchy things, but overhauling and rebuilding the entire lighting system is a complicated task, and since the current system worked without exploding and destroying everything: there was no reason to spend the time on it when that time could be better spent say, rearranging the entire data-pipeline within the game.
The only issue I see (I didn't spend long looking at it, so forgive me if I missed something) with your lighting system is that if the chunks above are not loaded, then the skylight calculator will think that the chunks below are open to the sky regardless of what was in the chunks above. This could, for example, break cave worlds and caverns, which is ~not good~. The conclusion on the CC thread was that a height-map system, storing the y coordinates of the highest known solid block at any x/y coordinate, was the best solution to this problem.
You could always simplify and use the solid/empty system to shortcut in some places;
if (isEmpty) { //create empty thing on heightmap } else if (isSolid) { //create solid thing on heightmap } else { //loop over every column individually (or array-erize the empty columns during generation?) };But w/e. It's a bit of a change of pace, but I also think you are too modest in the thread title; CC will actually IMPROVE performance and REDUCE lag.
I believe in the Invisible Pink Unicorn, bless her Invisible Pinkness.
-
View User Profile
-
View Posts
-
Send Message
Curse PremiumThis can be a problem, but only when the overhead chunks haven't been generated yet. And there's not much I can do about that without actually generating the chunks. If the overhead chunks are generated, but not loaded, then this is not a problem. I use a fancy version of a height map (look at the LightIndex classes) to solve this problem.
Ah, I overlooked the LightIndex stuff. I should have looked at the source more
The generation thing is indeed a problem, and sadly it's as you say; the only solution is to generate the world higher :'(
EDIT: wow that is pretty fancy ^^
EDIT 2:
A problem that faced the old CC mod, and Bartek's attempt at a CC mod, was the vertical chunk loading speed. In fact, since the player can fall at about 80 blocks, or 5 chunks, per second; it is very easy for the player to fall into unloaded space no matter how efficient the system is, especially since you may need to load up to 10, or even 20 chunks per second to keep them falling without interruption if they are at the edge or corner of a chunk. How do you plan on dealing with this particular issue?
I believe in the Invisible Pink Unicorn, bless her Invisible Pinkness.
-
View User Profile
-
View Posts
-
Send Message
Curse PremiumThat's a tricky issue. The only way I can deal with it is generate some tall terrains and do some testing to see what works and what doesn't. I need to get a feel for the current loading speed first before I can decide what measures are needed to improve loading speed. And before I can do that, I need to generate tall worlds. And before that, I need to remove the height limits.
We're getting there. =)
The CC thread mentions that the falling would pause while the chunks load(the speed would likely be the same when it unpaused, wouldn't be to hard to do). Alternatively it would act like the horizontal chunk problems.
The pausing is likely to be the best solution, combined with a slight cap to the max falling speed and so on.
The only thing to note is that where possible, it may be better to pause for longer and load many more chunks below the falling entity/player, since I think it would be much less annoying to get stopped 1 time for a 5 seconds than it would be to get stopped 5 times for 1 second each.
As for the horizontal chunk problems -> Technically I think the game does try to pause the entity until the chunk is loaded. The bigger problem there is that you usually only reach the edge of the loaded world when flying in creative mode, or when the chunks are corrupted in some way. Unfortunately, it is super easy to jump off of something tall in MC, and the last thing you want to do is poo on every player who wants to fall more than 99 blocks in a single jump. This means that there simply has to be a reliable system in place to deal with fast moving entities.
I believe in the Invisible Pink Unicorn, bless her Invisible Pinkness.
also you should check out Link Removed
-
View User Profile
-
View Posts
-
Send Message
Curse PremiumIt's no joke. =)
The joke is on us, and Mojang, for not being as awesome as Cuchaz
I raised the issue under the assumption that this would already be occurring. Even when the fall gets paused, it would only be to load additional chunks immediately below the player so that the fall can continue. The real problem is quite simply that;
Sadly, we can't just say "meh, the player usually doesn't fall along the intersection line of 4 chunks, so we can just pretend it will never happen", because then the game might simply end up exploding whenever a player does it. In fact, all that would really need to happen for you to fall out of the world, even on a lightning fast system, would be for your disk to hang or delay a read operation for a few ms.
Of course, using a linear system, this isn't a problem, because the chances of you, for example, randomly accelerating to an x speed of 100 blocks per second, are extremely low. Takes an extra second to load up the chunk? Not a problem, because it takes an extra 10 seconds to ~reach~ the chunk.
However, the chances of falling off of a 200 block high structure and hitting the peak speed of 80 blocks per second, are actually quite high. This makes it much easier to fall out of the loaded world, and for this reason, any CC mod needs some reliable system in place to ensure that the game does not poo itself.
I believe in the Invisible Pink Unicorn, bless her Invisible Pinkness.
Still, taller worlds negates that fact, but 200 blocks will not get you to terminal velocity. Just wanted to mention that.
also you should check out Link Removed