I have only region folder when i create new saves, and i can't unpack it with Conversion Utility... The save itself is working, and it save things :biggrin.gif: (i mean blocks i destroy etc.)
@EDIT
Nevermind, i used new Converter, and it works fine, thanks :smile.gif:
For Minecraft, we set up a rough schedule covering the major points that need to be added. For new content, we decided not to list it in detail, as that should be added in a more agile way where you kinda go from day to day deciding what would make the most sense, or just if someone gets a great idea that they want to try right away. The first two big points were, in order of priority, “New save file format”, and “Add mod support”.
For Minecraft, we set up a rough schedule covering the major points that need to be added. For new content, we decided not to list it in detail, as that should be added in a more agile way where you kinda go from day to day deciding what would make the most sense, or just if someone gets a great idea that they want to try right away. The first two big points were, in order of priority, “New save file format”, and “Add mod support”.
On twitter Notch said the new save file format was going to be a single file, which would probably be worse than the current situation.
For Minecraft, we set up a rough schedule covering the major points that need to be added. For new content, we decided not to list it in detail, as that should be added in a more agile way where you kinda go from day to day deciding what would make the most sense, or just if someone gets a great idea that they want to try right away. The first two big points were, in order of priority, “New save file format”, and “Add mod support”.
On twitter Notch said the new save file format was going to be a single file, which would probably be worse than the current situation.
On most filesystems it should make little difference whether it's many small files vs. one large file, if read/write is coded properly into Minecraft.
On most filesystems it should make little difference whether it's many small files vs. one large file, if read/write is coded properly into Minecraft.
Incorrect, regardless of how well cached a file is there will be some measurable read/write performance when the system does make a disk request. Trying to read and and write to thousands of small files is the worst case scenario for mechanical hard drives because of seek times, and this results in abysmal performance.
Primary:
1) Performance. Read/write delay should be minimal. The main way to accomplish this is high spatial locality in order to reduce disk seeks-- that is, if you read the data from one chunk, the data from adjacent chunks is nearby, and quick to access.
2) Reliability. If minecraft crashes, as little as possible should be lost. Corruption is unacceptable.
3) Efficiency. Save files should waste as little disk space as possible. Compression is a first step towards this goal.
Secondary:
4) Compatibility. Old saves should not stop working if a new save system is devised.
5) Simplicity. Many people write code that interfaces with Minecraft's world data. A simpler format makes this easier for them.
6) Portability. Saves should be easy and fast to copy, share, backup, and restore.
Current system:
-1 Performance. Has a lot of room for improvement.
+2 Reliability. Good. Only individual chunks may be lost.
-3 Efficiency. Storing metadata for every chunk wastes filesystem resources.
=4 Compatibility. Not applicable
+5 Simplicity. The simplest way possible.
-6 Portability. Copying thousands of files is slow. Chunk modifications can be detected with high granularity from file modified timestamps.
McRegion:
+1 Performance. Chunks are stored in regions for high spatial locality on disk.
+2 Reliability. A crash will lose at most one chunk.
+3 Efficiency. Chunks are packed in groups of 1024 for less wasteful metadata. Region file fragmentation is little-to-zero in practice.
+4 Compatibility. New formats for chunk storage can be added easily, using the version field. This was demonstrated with McRegion v5, which changed the default compression from GZIP to Deflate without issues.
=5 Simplicity. Pretty simple, but complicates things beyond the base case.
+6 Portability. Copying a few dozen files is fast. Chunk modifications are detected with reduced granularity, but chunks are usually modified in large groups regardless.
Single file:
=1 Performance. Unless there's a good way to ensure spatial locality, it is likely that the performance of a single file will be closer to that of thousands of chunks than McRegion.
=2 Reliability. Variable depending on implementation. Worst case: a crash corrupts an entire world''s save file.
-3 Efficiency. Unless good fragmentation handling protocols are in place, wasted space will be a major issue. The problem of efficient allocation has been studied for decades by filesystem designers.
=4 Compatibility. Unknown, depends on implementation.
--5 Simplicity. Placing all files for a world into a single binary format, possibly with complex data structures in place to mitigate performance/reliability/efficiency concerns, increases complexity of read-only tools (cartography), and greatly increases the complexity of read-write tools (INVEdit, McEdit, various world generation tools, etc.)
=6 Portability. Copying a single file is very fast. Unfortunately, copying only changes chunks becomes much more difficult, requiring binary differentials of segments of the file.
I believe these evaluations are only slightly biased towards my own work. Also note that in the common case of someone sharing a world save over the internet, it is likely that they will first compress it with ZIP/RAR/7ZIP for smaller size, rendering most qualifications regarding efficiency moot. Thus, the primary advantage of a single file format: the simple fact that a single file contains all the necessary data, meaning sharing operations require only a single file transfer, is useless for the internet where an additional compressed container will be applied regardless.
tl;dr: a single file save format is not worth the time and effort
...
[very useful discussion of pros & cons of save file formats]
...
tl;dr: a single file save format is not worth the time and effort
Thanks for this information Scaevolus. I'm wondering what you think about very large worlds? (in excess of 1 million chunks). Does MCregion scale well with world size? I'm assuming it should do just as good as the current situation since it's still a granular solution, just fewer files. However, a single-file of a million chunk world sounds a bit worrisome (due to having all your eggs in one basket and complicating chunk recovery, both of which you mentioned in your post).
Do you have any comments on this? Am I worrying too much that my SMP world of 1 million+ chunks will be severely compromised by a single-file system? Would a solution like MCregion work better for very large worlds compared to either the current situation or single-file?
Thanks for this information Scaevolus. I'm wondering what you think about very large worlds? (in excess of 1 million chunks). Does MCregion scale well with world size? I'm assuming it should do just as good as the current situation since it's still a granular solution, just fewer files. However, a single-file of a million chunk world sounds a bit worrisome (due to having all your eggs in one basket and complicating chunk recovery, both of which you mentioned in your post).
Do you have any comments on this? Am I worrying too much that my SMP world of 1 million+ chunks will be severely compromised by a single-file system? Would a solution like MCregion work better for very large worlds compared to either the current situation or single-file?
Thanks again.
A single file save would probably give decreased performance as the number of chunks increases. McRegion scales well with world size, although it may have more files in a single directory (region/) than the maximum number of chunks in a single directory, since chunks are split between 4096 directories, and region files store 1024 chunks each. Modern filesystems do not experience noticeable slowdown from a few thousand files in a directory, so I do not believe this is an issue.
I agree that a single-file save will almost certainly have bad performance for large worlds. McRegion works better for very large worlds than either of the alternatives, since it avoids the overhead of having the filesystem track millions of files -- it would have perhaps a few thousand region files-- and the fragmentation problems of a single file save.
If you do decide to move your world to McRegion, I would be interested in the number of chunks compared to the number of regions.
So basically what this does is stores all the saves to a single large file in the RAM and dumps it into the HDD? Exactly how large would that file be? I have 8GB of RAM so it being ~50-500MB wouldn't be bad. But this is great for my SSD. Minecraft is probably the number one game for killing SSD's with the huge amount of small file writes. :sad.gif:
Hey, I'm also crashing with a SHA1 digest error for dd.class.
I deleted META-INF in minecraft.jar. repeat: I deleted META-INF in minecraft.jar.
After I run minecraft, it crashes and a new META-INF is generated in minecraft.jar .
Any ideas?
--- BEGIN ERROR REPORT a1dce528 --------
Generated 1/7/11 4:49 PM
Minecraft: Minecraft Beta 1.1_02
OS: Mac OS X (i386) version 10.6.4
Java: 1.6.0_22, Apple Inc.
VM: Java HotSpot(TM) Client VM (mixed mode), Apple Inc.
LWJGL: 2.4.2
OpenGL: Intel GMA X3100 OpenGL Engine version 2.0 APPLE-1.6.18, Intel Inc.
java.lang.SecurityException: SHA1 digest error for dd.class
at sun.security.util.ManifestEntryVerifier.verify(ManifestEntryVerifier.java:192)
at java.util.jar.JarVerifier.processEntry(JarVerifier.java:212)
at java.util.jar.JarVerifier.update(JarVerifier.java:199)
at java.util.jar.JarVerifier$VerifierStream.read(JarVerifier.java:449)
at sun.misc.Resource.getBytes(Resource.java:108)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:257)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at lv.a(SourceFile:34)
at br.a(SourceFile:91)
at net.minecraft.client.Minecraft.a(SourceFile:480)
at dp.a(SourceFile:187)
at br.a(SourceFile:69)
at br.e(SourceFile:116)
at br.d(SourceFile:104)
at net.minecraft.client.Minecraft.i(SourceFile:1073)
at net.minecraft.client.Minecraft.run(SourceFile:677)
at java.lang.Thread.run(Thread.java:680)
--- END ERROR REPORT 84b5ebcf ----------
So basically what this does is stores all the saves to a single large file in the RAM and dumps it into the HDD? Exactly how large would that file be? I have 8GB of RAM so it being ~50-500MB wouldn't be bad. But this is great for my SSD. Minecraft is probably the number one game for killing SSD's with the huge amount of small file writes. :sad.gif:
How did you get that impression? This mod doesn't store files in RAM. It just changes how they're laid out on the disk.
-
View User Profile
-
View Posts
-
Send Message
Retired StaffTHIS MOD IS INCREDIBLY ****ING AWSOME!!!
CDR's PTC HD Infinity [128x Texture Pack]
How to find the minecraft.jar file
Modifying the minecraft.jar file
One Stop Mod Shop
Ill check
EDIT
Yep that was the Problem sorry for the confusion someone had called my attention when I was clicking the download link.
@EDIT
Nevermind, i used new Converter, and it works fine, thanks :smile.gif:
On twitter Notch said the new save file format was going to be a single file, which would probably be worse than the current situation.
On most filesystems it should make little difference whether it's many small files vs. one large file, if read/write is coded properly into Minecraft.
Incorrect, regardless of how well cached a file is there will be some measurable read/write performance when the system does make a disk request. Trying to read and and write to thousands of small files is the worst case scenario for mechanical hard drives because of seek times, and this results in abysmal performance.
Primary:
1) Performance. Read/write delay should be minimal. The main way to accomplish this is high spatial locality in order to reduce disk seeks-- that is, if you read the data from one chunk, the data from adjacent chunks is nearby, and quick to access.
2) Reliability. If minecraft crashes, as little as possible should be lost. Corruption is unacceptable.
3) Efficiency. Save files should waste as little disk space as possible. Compression is a first step towards this goal.
Secondary:
4) Compatibility. Old saves should not stop working if a new save system is devised.
5) Simplicity. Many people write code that interfaces with Minecraft's world data. A simpler format makes this easier for them.
6) Portability. Saves should be easy and fast to copy, share, backup, and restore.
Current system:
-1 Performance. Has a lot of room for improvement.
+2 Reliability. Good. Only individual chunks may be lost.
-3 Efficiency. Storing metadata for every chunk wastes filesystem resources.
=4 Compatibility. Not applicable
+5 Simplicity. The simplest way possible.
-6 Portability. Copying thousands of files is slow. Chunk modifications can be detected with high granularity from file modified timestamps.
McRegion:
+1 Performance. Chunks are stored in regions for high spatial locality on disk.
+2 Reliability. A crash will lose at most one chunk.
+3 Efficiency. Chunks are packed in groups of 1024 for less wasteful metadata. Region file fragmentation is little-to-zero in practice.
+4 Compatibility. New formats for chunk storage can be added easily, using the version field. This was demonstrated with McRegion v5, which changed the default compression from GZIP to Deflate without issues.
=5 Simplicity. Pretty simple, but complicates things beyond the base case.
+6 Portability. Copying a few dozen files is fast. Chunk modifications are detected with reduced granularity, but chunks are usually modified in large groups regardless.
Single file:
=1 Performance. Unless there's a good way to ensure spatial locality, it is likely that the performance of a single file will be closer to that of thousands of chunks than McRegion.
=2 Reliability. Variable depending on implementation. Worst case: a crash corrupts an entire world''s save file.
-3 Efficiency. Unless good fragmentation handling protocols are in place, wasted space will be a major issue. The problem of efficient allocation has been studied for decades by filesystem designers.
=4 Compatibility. Unknown, depends on implementation.
--5 Simplicity. Placing all files for a world into a single binary format, possibly with complex data structures in place to mitigate performance/reliability/efficiency concerns, increases complexity of read-only tools (cartography), and greatly increases the complexity of read-write tools (INVEdit, McEdit, various world generation tools, etc.)
=6 Portability. Copying a single file is very fast. Unfortunately, copying only changes chunks becomes much more difficult, requiring binary differentials of segments of the file.
I believe these evaluations are only slightly biased towards my own work. Also note that in the common case of someone sharing a world save over the internet, it is likely that they will first compress it with ZIP/RAR/7ZIP for smaller size, rendering most qualifications regarding efficiency moot. Thus, the primary advantage of a single file format: the simple fact that a single file contains all the necessary data, meaning sharing operations require only a single file transfer, is useless for the internet where an additional compressed container will be applied regardless.
tl;dr: a single file save format is not worth the time and effort
"%ProgramFiles(x86)%\Java\jre6\bin\java.exe" -jar RegionTool.jar unpack "C:\whatever path"
or
java -jar RegionTool.jar unpack "C:\whateverpathitis"
Thanks for this information Scaevolus. I'm wondering what you think about very large worlds? (in excess of 1 million chunks). Does MCregion scale well with world size? I'm assuming it should do just as good as the current situation since it's still a granular solution, just fewer files. However, a single-file of a million chunk world sounds a bit worrisome (due to having all your eggs in one basket and complicating chunk recovery, both of which you mentioned in your post).
Do you have any comments on this? Am I worrying too much that my SMP world of 1 million+ chunks will be severely compromised by a single-file system? Would a solution like MCregion work better for very large worlds compared to either the current situation or single-file?
Thanks again.
A single file save would probably give decreased performance as the number of chunks increases. McRegion scales well with world size, although it may have more files in a single directory (region/) than the maximum number of chunks in a single directory, since chunks are split between 4096 directories, and region files store 1024 chunks each. Modern filesystems do not experience noticeable slowdown from a few thousand files in a directory, so I do not believe this is an issue.
I agree that a single-file save will almost certainly have bad performance for large worlds. McRegion works better for very large worlds than either of the alternatives, since it avoids the overhead of having the filesystem track millions of files -- it would have perhaps a few thousand region files-- and the fragmentation problems of a single file save.
If you do decide to move your world to McRegion, I would be interested in the number of chunks compared to the number of regions.
sigs are dumb
I deleted META-INF in minecraft.jar. repeat: I deleted META-INF in minecraft.jar.
After I run minecraft, it crashes and a new META-INF is generated in minecraft.jar .
Any ideas?
How did you get that impression? This mod doesn't store files in RAM. It just changes how they're laid out on the disk.
World's End? Ultra FPS on a laptop? Or complete 1 FPS slaughter?
They should both help performance a little bit.
They work together. Did you download the server version instead of the client version?
(Why would I make two mods that I can't use at the same time?)