I have already playtested it and got it working with Tekkit 1.2.5, but How would you guys like a multi-threaded Server jar? Everybody said that it was impossible, but I've gotten a somewhat buggy working version that had improved overall server stability and reliability of minecraft Server ticks by 200-10,000%. This number, of course, was tested using a basic setup in tekkit that lags out even the heartiest servers to 2Tps of 20Tps.
The reason why everybody says that it is impossible is that it requires at least 500-1000 hours of programming labor to modify tons of minecraft jar files to support multithreading.
Why did it get a 10,000% increase? I separated the server's main single-threaded tick system into about 30-50 different little threads that each are synchronized in order with the minecraft jar.
But basically, what it does is split the server tick thread up into multiple sections and ticks for each section (about 30-60 sections) on a synchronized timer. This does go over the main issue of looping through 65,000,000 TileEntities. It also makes sure that players are always connected even if there are massive amounts of lag. Unfortunately, nobody has supported me in my quest to make Tekkit... and Minecraft in general, faster.
Features:
Possible Use of Spigot as Base (will need to get any licensing or whatever is required)
Stability - Impossible to crash using any plugins or mods (Besides WorldEdit lol)
Internal Chunk Unloading - Unload all of those fluffy chunks that are using up so much space on your server!
Internal Log Splitting and Filtering - Tired of all of that fluff in the console window? Easily fixed!
Ticking for Each player instead of each World - Each player gets their own "special thread" which manages each player's "corner of the world". It will tick for 2x2-16x16 chunks. This thread may also tick for networking, unknown territory. This will prevent the server from ticking all 1,000-30,000 chunks that are currently loaded by the server.
Adjustable Max Ticks / Sec - Enable your server to go at 40 Ticks per second!
Managed High-speed Event handling - Bukkit events have become a bit complicated when multithreading was introduced. For the event system has to be self-managed by a single thread and all information is handled via a pass-through system.
Primary Ticks Threading - As of now, there is a thread for Networking, 6 threads for each world, a-thread-per forge mod, a thread for plugin's Heartbeat tick and multiple underlying threads for player management.
Max World Height Increased to 8086! (Note that insane world heights require the use of more CPU and sace)
Possible new APIs for mod/plugin Developers for profiling and tick speed management
Built-in Spout plugin (Lets support Spout!)
Possible new "FastDownload" add-in that sort of works like GMod's FastDownload. Possibly implemented with Forge to automatically download the required mods from the server or a website(s). (Will probably require a modification of the client)
Internal Profiling for server admins to find that "Hot spot" where your mods or plugins are literally killing your server. (This will look through using TileEntities, Chunks, Entities, Threads, Methods, ect.)
Versions
Bukkit or non Bukkit versions
Forge or non Forge versions (+ForgeIRC, ect)
Minecraft Versions 1.2.5, 1.3.1, 1.4.6
Base Requirements:
At least 2 Server CPU Cores, the more cores, the better. (uses a bit more cpu then normal, but the extra cpu used never increases. However, when you get a bigger server, the dynamic CPU usage will increase)
I am going to naturally support the FTB (Feed the Beast) and Tekkit modpacks and several other adaptations to the system to help improve overall performance.
Please give me feedback on your opinions of a continuation of this Distribution of MC.
No, it doesn't. CB++ was mostly modifications for several things such as leaf decay, ect. Spigot is a modification of CBpp and it does not support Multithreading either. No server jar Distro that I've seen so far uses multithreading on the main server Thread.
* Cough, add in some sort of profiling option like bukkit on the entities and I will personally support you all the way.
We often see FTB servers etc... using 130% cpu usage with no reason or explanation anywhere.
Yes, there is nothing that really profiles TileEntities atm Because nobody has really thought of it. I guess I could add in a command that is built-in and returns the # of TileEntities being looped through Every tick and which chunks have them, nearest players, ect. Alongside normal entities and a basic API for the entity system that people can leach off of for use with plugins, ect.
I have made a profiling option on TekkitRestrict that returns a file 'threadinfo.txt' when you use the command "/tr threadlag". I guess this could also be an alternative to the method above to where you would run the command "/tilelag and /entitylag" to generate (fully) profiled files that can be pretty long based on the current Chunk. Like... the "Top Chunk" list or somehting that uses the most # of tileEntities.
I'm taking it one step further, I've added a full internal profiler for Minecraft to the todo list!
As for now, I have a playtest demo, but only with Tekkit 1.2.5, as it was originally designed for Tekkit and all it's laggyness. Dont worry, it will support all of the newer versions in the future.
RobinThreads (Split up server thread) running at minimal
This sounds great. I can't believe the current servers still don't support multiple cores. It's so frustrating having low TPS when a bunch of cores are sitting idle.
Hes modifying the SERVER not the game client.
Minecraft DOES NOT multithread properly AT ALL.
I have an 8 thread (4 cores) dedicated server and my minecraft server will never use more than 2 threads (1 CORE). This is damn well needed. (PS the client barely multithreads either , it only ever uses one of my 4 CPU cores at home)
I support this guy all the way. Lets hope we see a build soon.
Reader? Read what. All you did was say that the client does not need to be modified at all. So please elaborate on your statement, rather than just throwing it out there, and then just blatantly insulting me when i pointed the fact that it does not multi thread to you. I think enough to run a popular Minecraft server and tell you that neither craftbukkit or the standard MC server multithread.
The client isn't the issue here, and specifically, should you even want to improve client performance you wouldn't simply "use more threads"- how about using something like Rootbeer to access the GPU instead of using as bad as a library as LWJGL...
Anyway, I don't think you understand the performance issue. A new thread is spawned per client. This is why people think a single player uses "300KB" memory, the default heap size for a thread is 256KB on Windows JVM and 512KB on *nix JVM. If you simply LOOK AT THE CODE, you can clearly see a player actually has under 2KB of information tagged along to itself, the rest are CHUNKS and are not dependencies of a player. Even so, chunks are not too memory-heavy either.
To improve the performance (and scalability) of Minecraft servers you can use a "special trick" called reactor-based networking. Java offers an implementation of this with NIO. It works by assigning, instead of a thread, a key, and when data is read/write it is done asynchronously (and hence when you read/write in code you aren't actually transferring the data just yet), instead it is buffered- queued. From the data read, by keys the data is shipped to its handlers, which, as with writing, also is asynchronous and actually reading after the data was actually transferred. What's so excellent about this is you can also have asynchronous client accepts, hence, you're able to use a cycle for accepts... so, considering Minecraft is a cycled game you don't increase client response time. Also, you gain the ability to throttle accepts and hence have a "login queue" (think League of Legends) without actually having to write the code to do this. You can also improve accept time with a thread pool, but this simply isn't needed for Minecraft because they usually only have as many as 2000 players. As well, you don't need a thread pool for read/writes... it isn't expensive to use a buffer, simple as that, and read/writes must be blocking anyway with TCP.
Not everyone will be able to do this, and not everyone uses a *nix based host.
For example someone running a server from their computer say on windows 7.
Thus having a system which is able to multi thread would give a simple increase for people not able to use that, what your saying is that because Minecraft is badly designed the whole thing should be thrown out the window and nothing attempted otherwise until it is (It sounds like that anyway, i am not implying you mean it)
However i shall look into this reactor based networking, i am always looking to improve the performance of my server, which at the moment is limited due to the fact it will only use 1 cores worth of the i7 it runs on. If you have any links you could send me on the topic it would be much appreciated.
As I said before, not much of a reader, are you? Your reply makes it seem like you either (1) completely skipped over the critical information I had presented or (2) didn't understand the information.
As I said before, not much of a reader, are you? Your reply makes it seem like you either (1) completely skipped over the critical information I had presented or (2) didn't understand the information.
Wow. Just wow. I'm coming here late, following links from other threads, but I gotta say you don't understand half what you think you do.
Minecraft, both client and server, is written in Java. NIO has zero to do with the issue of Minecraft running on a Java VM not using all available processing power.
When modifying a source to utilize processing power not factored into the initial design, the amount of effort required is variable, depending on initial design and implementation of classes.
Simply branching threads willy-nilly can have unintended consequences, some catastrophic. Changing the state of an object designed for a single thread design can be catastrophic (or "buggy") when multiple threads have access.
NIO simply addresses a failing in Java regarding input/output. It does not address the inability of a software package to utilize multiple threads to make best use of available processing power.
If you understood what NIO in Java actually delivers you wouldn't have posted in this thread. If you understood what the OP was attempting, you wouldn't have posted.
Before you call others "newbies," ask yourself if your post is intended to "prop your ego" or "share information that might save someone else time and labor."
If no information exists in your post that might save someone time and labor, your post is "trolling," and not helpful. This post is also an example of trolling.
I'm not propping my ego but I am pointing out your comments are the output of a true newb who doesn't understand the issue before posting a negative comment - dropping whatever they might have learned in the last week as if it solves the original problem without the slightest understanding of the OP's post. Arguably, I am saving the OP the hassle of investigating NIO and realizing it doesn't address the issues. But my main thrust is discounting the input of someone who doesn't understand the problem but insists on insulting people who do.
In point of fact, your post is irrelevant - a mistake only a newb would make. NIO could help with input/output, but the underlying issue is Minecraft running under Java doesn't make full use of the processing power available in a world of multi-core, multi-processor architecture.
I respect the OP's endeavor. I might suggest porting Minecraft server to C++ would solve a whole host of Java-centric performance issues while still allowing multi-architecture compatibility. I admire the OP's intent.
Mojang really needs to learn C++ and put the game there. Drop Java except for clients, IMHO.
I respect the OP's endeavor. I might suggest porting Minecraft server to C++ would solve a whole host of Java-centric performance issues while still allowing multi-architecture compatibility. I admire the OP's intent.
Mojang really needs to learn C++ and put the game there. Drop Java except for clients, IMHO.
I apologize to the OP for cluttering his thread.
Well. I so far haven't heard of a linux working C++ build, so correct me if i'm wrong. But the main point of minecraft clients AND servers is to be cross-compatible. If you switch to C++, then i would assume that it would then only be available to Windows, and (possibly) Mac. So I would think that OP's project is the way to go.
P.S: OP, try and scratch together a team of developers to help you with the project.
Rollback Post to RevisionRollBack
Free + Crabs + Ability to trample/suffocate opponents in Cortex Command = Free Bombs.
It is my understanding that the 360 version of Minecraft is in C++, so it has been done successfully and profitably. I've played it on the 360 the most, and can say that performance is generally equivalent to the game on my PC, but lag on the PC is usually an intermittent, temporary issue. The 360 has a tiny fraction of the memory available to the PC version, hence the limit on world size, but worlds are still huge and the game runs smooth and lag-free for the most part. (I built a lag-machine with redstone and pistons on the Xbox one day.)
I realize the game is far enough along that porting to C++ for the PC is not practical/never gonna happen. I was just commenting that many of the performance issues in Minecraft are a result of Java, not ignorant programmers at Mojang. Trying to fix the problem within Java is admirable, and I'm interested to see where it goes. But refactoring the game in C++ would address numerous failings in one fell swoop.
It is my understanding that the 360 version of Minecraft is in C++, so it has been done successfully and profitably. I've played it on the 360 the most, and can say that performance is generally equivalent to the game on my PC, but lag on the PC is usually an intermittent, temporary issue. The 360 has a tiny fraction of the memory available to the PC version, hence the limit on world size, but worlds are still huge and the game runs smooth and lag-free for the most part. (I built a lag-machine with redstone and pistons on the Xbox one day.)
I realize the game is far enough along that porting to C++ for the PC is not practical/never gonna happen. I was just commenting that many of the performance issues in Minecraft are a result of Java, not ignorant programmers at Mojang. Trying to fix the problem within Java is admirable, and I'm interested to see where it goes. But refactoring the game in C++ would address numerous failings in one fell swoop.
I would like to restate that Notch himself claimed that he used java for cross-compatibility. Which is one of the reasons why minecraft is so popular. But in terms of game speed/smoothness, i have no doubt that C++ would make the game somewhat smoother.
Rollback Post to RevisionRollBack
Free + Crabs + Ability to trample/suffocate opponents in Cortex Command = Free Bombs.
Please do this Its so frustrating having an 8 threaded CPU on my dedicated server, yet Minecraft only ever using about 1.8 threads and wasting the rest!
TerraCore Minecraft Multithreading
Why have this? Look here for some TPS problems with mods in Tekkit. This goes for almost every single mod.
I have already playtested it and got it working with Tekkit 1.2.5, but How would you guys like a multi-threaded Server jar? Everybody said that it was impossible, but I've gotten a somewhat buggy working version that had improved overall server stability and reliability of minecraft Server ticks by 200-10,000%. This number, of course, was tested using a basic setup in tekkit that lags out even the heartiest servers to 2Tps of 20Tps.
The reason why everybody says that it is impossible is that it requires at least 500-1000 hours of programming labor to modify tons of minecraft jar files to support multithreading.
Why did it get a 10,000% increase? I separated the server's main single-threaded tick system into about 30-50 different little threads that each are synchronized in order with the minecraft jar.
But basically, what it does is split the server tick thread up into multiple sections and ticks for each section (about 30-60 sections) on a synchronized timer. This does go over the main issue of looping through 65,000,000 TileEntities. It also makes sure that players are always connected even if there are massive amounts of lag. Unfortunately, nobody has supported me in my quest to make Tekkit... and Minecraft in general, faster.
Features:
Please give me feedback on your opinions of a continuation of this Distribution of MC.
No, it doesn't. CB++ was mostly modifications for several things such as leaf decay, ect. Spigot is a modification of CBpp and it does not support Multithreading either. No server jar Distro that I've seen so far uses multithreading on the main server Thread.
Yes, there is nothing that really profiles TileEntities atm Because nobody has really thought of it. I guess I could add in a command that is built-in and returns the # of TileEntities being looped through Every tick and which chunks have them, nearest players, ect. Alongside normal entities and a basic API for the entity system that people can leach off of for use with plugins, ect.
I have made a profiling option on TekkitRestrict that returns a file 'threadinfo.txt' when you use the command "/tr threadlag". I guess this could also be an alternative to the method above to where you would run the command "/tilelag and /entitylag" to generate (fully) profiled files that can be pretty long based on the current Chunk. Like... the "Top Chunk" list or somehting that uses the most # of tileEntities.
I'm taking it one step further, I've added a full internal profiler for Minecraft to the todo list!
As for now, I have a playtest demo, but only with Tekkit 1.2.5, as it was originally designed for Tekkit and all it's laggyness. Dont worry, it will support all of the newer versions in the future.
RobinThreads (Split up server thread) running at minimal
I can hardly wait. I mean, chunk unloading? This is good to hear!
Followed and bookmarked.
If there is any way I can help, shoot me a PM.
As a friend, of course.
Just use NIO... you don't need to modify the client at all.
Just a developer :-)
Just a developer :-)
Anyway, I don't think you understand the performance issue. A new thread is spawned per client. This is why people think a single player uses "300KB" memory, the default heap size for a thread is 256KB on Windows JVM and 512KB on *nix JVM. If you simply LOOK AT THE CODE, you can clearly see a player actually has under 2KB of information tagged along to itself, the rest are CHUNKS and are not dependencies of a player. Even so, chunks are not too memory-heavy either.
To improve the performance (and scalability) of Minecraft servers you can use a "special trick" called reactor-based networking. Java offers an implementation of this with NIO. It works by assigning, instead of a thread, a key, and when data is read/write it is done asynchronously (and hence when you read/write in code you aren't actually transferring the data just yet), instead it is buffered- queued. From the data read, by keys the data is shipped to its handlers, which, as with writing, also is asynchronous and actually reading after the data was actually transferred. What's so excellent about this is you can also have asynchronous client accepts, hence, you're able to use a cycle for accepts... so, considering Minecraft is a cycled game you don't increase client response time. Also, you gain the ability to throttle accepts and hence have a "login queue" (think League of Legends) without actually having to write the code to do this. You can also improve accept time with a thread pool, but this simply isn't needed for Minecraft because they usually only have as many as 2000 players. As well, you don't need a thread pool for read/writes... it isn't expensive to use a buffer, simple as that, and read/writes must be blocking anyway with TCP.
Just a developer :-)
Regardless, here's a tutorial for implementation reactor-based networking with Java NIO:
http://cs.brown.edu/...s/j-nio-ltr.pdf
Just a developer :-)
Wow. Just wow. I'm coming here late, following links from other threads, but I gotta say you don't understand half what you think you do.
Minecraft, both client and server, is written in Java. NIO has zero to do with the issue of Minecraft running on a Java VM not using all available processing power.
When modifying a source to utilize processing power not factored into the initial design, the amount of effort required is variable, depending on initial design and implementation of classes.
Simply branching threads willy-nilly can have unintended consequences, some catastrophic. Changing the state of an object designed for a single thread design can be catastrophic (or "buggy") when multiple threads have access.
NIO simply addresses a failing in Java regarding input/output. It does not address the inability of a software package to utilize multiple threads to make best use of available processing power.
If you understood what NIO in Java actually delivers you wouldn't have posted in this thread. If you understood what the OP was attempting, you wouldn't have posted.
Before you call others "newbies," ask yourself if your post is intended to "prop your ego" or "share information that might save someone else time and labor."
If no information exists in your post that might save someone time and labor, your post is "trolling," and not helpful. This post is also an example of trolling.
I'm not propping my ego but I am pointing out your comments are the output of a true newb who doesn't understand the issue before posting a negative comment - dropping whatever they might have learned in the last week as if it solves the original problem without the slightest understanding of the OP's post. Arguably, I am saving the OP the hassle of investigating NIO and realizing it doesn't address the issues. But my main thrust is discounting the input of someone who doesn't understand the problem but insists on insulting people who do.
In point of fact, your post is irrelevant - a mistake only a newb would make. NIO could help with input/output, but the underlying issue is Minecraft running under Java doesn't make full use of the processing power available in a world of multi-core, multi-processor architecture.
I respect the OP's endeavor. I might suggest porting Minecraft server to C++ would solve a whole host of Java-centric performance issues while still allowing multi-architecture compatibility. I admire the OP's intent.
Mojang really needs to learn C++ and put the game there. Drop Java except for clients, IMHO.
I apologize to the OP for cluttering his thread.
Well. I so far haven't heard of a linux working C++ build, so correct me if i'm wrong. But the main point of minecraft clients AND servers is to be cross-compatible. If you switch to C++, then i would assume that it would then only be available to Windows, and (possibly) Mac. So I would think that OP's project is the way to go.
P.S: OP, try and scratch together a team of developers to help you with the project.
Free + Crabs + Ability to trample/suffocate opponents in Cortex Command = Free Bombs.
-
I realize the game is far enough along that porting to C++ for the PC is not practical/never gonna happen. I was just commenting that many of the performance issues in Minecraft are a result of Java, not ignorant programmers at Mojang. Trying to fix the problem within Java is admirable, and I'm interested to see where it goes. But refactoring the game in C++ would address numerous failings in one fell swoop.
I would like to restate that Notch himself claimed that he used java for cross-compatibility. Which is one of the reasons why minecraft is so popular. But in terms of game speed/smoothness, i have no doubt that C++ would make the game somewhat smoother.
Free + Crabs + Ability to trample/suffocate opponents in Cortex Command = Free Bombs.
Run 3 more servers........