How to read a binary file quickly in c#? (ReadOnlySpan vs MemoryStream)











up vote
3
down vote

favorite
1












I'm trying to parse a binary file as fastest as possible. So this is what I first tried to do:



using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
m.Position = 0;

using (BinaryReaderBigEndian b = new BinaryReaderBigEndian(m)) {
while (b.BaseStream.Position != b.BaseStream.Length) {
UInt32 value = b.ReadUInt32();
} } } } }


Where BinaryReaderBigEndian class is implemented as it follows:



public static class BinaryReaderBigEndian {
public BinaryReaderBigEndian(Stream stream) : base(stream) { }

public override UInt32 ReadUInt32() {
var x = base.ReadBytes(4);
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }


Then, I tried to get a performance improvement using ReadOnlySpan instead of MemoryStream. So, I tried doing:



using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
int position = 0;
ReadOnlySpan<byte> stream = new ReadOnlySpan<byte>(m.ToArray());

while (position != stream.Length) {
UInt32 value = stream.ReadUInt32(position);
position += 4;
} } } }


Where BinaryReaderBigEndian class changed in:



public static class BinaryReaderBigEndian {
public override UInt32 ReadUInt32(this ReadOnlySpan<byte> stream, int start) {
var data = stream.Slice(start, 4).ToArray();
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }


But, unfortunately, I didn't notice any improvement. So, where am I doing wrong?










share|improve this question







New contributor




heliosophist is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • where is your bottle neck? CPU, Memory, Disk access?
    – Richard Hubley
    2 hours ago










  • Nowhere! This is why I'm surprised.
    – heliosophist
    2 hours ago















up vote
3
down vote

favorite
1












I'm trying to parse a binary file as fastest as possible. So this is what I first tried to do:



using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
m.Position = 0;

using (BinaryReaderBigEndian b = new BinaryReaderBigEndian(m)) {
while (b.BaseStream.Position != b.BaseStream.Length) {
UInt32 value = b.ReadUInt32();
} } } } }


Where BinaryReaderBigEndian class is implemented as it follows:



public static class BinaryReaderBigEndian {
public BinaryReaderBigEndian(Stream stream) : base(stream) { }

public override UInt32 ReadUInt32() {
var x = base.ReadBytes(4);
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }


Then, I tried to get a performance improvement using ReadOnlySpan instead of MemoryStream. So, I tried doing:



using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
int position = 0;
ReadOnlySpan<byte> stream = new ReadOnlySpan<byte>(m.ToArray());

while (position != stream.Length) {
UInt32 value = stream.ReadUInt32(position);
position += 4;
} } } }


Where BinaryReaderBigEndian class changed in:



public static class BinaryReaderBigEndian {
public override UInt32 ReadUInt32(this ReadOnlySpan<byte> stream, int start) {
var data = stream.Slice(start, 4).ToArray();
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }


But, unfortunately, I didn't notice any improvement. So, where am I doing wrong?










share|improve this question







New contributor




heliosophist is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.




















  • where is your bottle neck? CPU, Memory, Disk access?
    – Richard Hubley
    2 hours ago










  • Nowhere! This is why I'm surprised.
    – heliosophist
    2 hours ago













up vote
3
down vote

favorite
1









up vote
3
down vote

favorite
1






1





I'm trying to parse a binary file as fastest as possible. So this is what I first tried to do:



using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
m.Position = 0;

using (BinaryReaderBigEndian b = new BinaryReaderBigEndian(m)) {
while (b.BaseStream.Position != b.BaseStream.Length) {
UInt32 value = b.ReadUInt32();
} } } } }


Where BinaryReaderBigEndian class is implemented as it follows:



public static class BinaryReaderBigEndian {
public BinaryReaderBigEndian(Stream stream) : base(stream) { }

public override UInt32 ReadUInt32() {
var x = base.ReadBytes(4);
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }


Then, I tried to get a performance improvement using ReadOnlySpan instead of MemoryStream. So, I tried doing:



using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
int position = 0;
ReadOnlySpan<byte> stream = new ReadOnlySpan<byte>(m.ToArray());

while (position != stream.Length) {
UInt32 value = stream.ReadUInt32(position);
position += 4;
} } } }


Where BinaryReaderBigEndian class changed in:



public static class BinaryReaderBigEndian {
public override UInt32 ReadUInt32(this ReadOnlySpan<byte> stream, int start) {
var data = stream.Slice(start, 4).ToArray();
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }


But, unfortunately, I didn't notice any improvement. So, where am I doing wrong?










share|improve this question







New contributor




heliosophist is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











I'm trying to parse a binary file as fastest as possible. So this is what I first tried to do:



using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
m.Position = 0;

using (BinaryReaderBigEndian b = new BinaryReaderBigEndian(m)) {
while (b.BaseStream.Position != b.BaseStream.Length) {
UInt32 value = b.ReadUInt32();
} } } } }


Where BinaryReaderBigEndian class is implemented as it follows:



public static class BinaryReaderBigEndian {
public BinaryReaderBigEndian(Stream stream) : base(stream) { }

public override UInt32 ReadUInt32() {
var x = base.ReadBytes(4);
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }


Then, I tried to get a performance improvement using ReadOnlySpan instead of MemoryStream. So, I tried doing:



using (FileStream filestream = path.OpenRead()) {
using (var d = new GZipStream(filestream, CompressionMode.Decompress)) {
using (MemoryStream m = new MemoryStream()) {
d.CopyTo(m);
int position = 0;
ReadOnlySpan<byte> stream = new ReadOnlySpan<byte>(m.ToArray());

while (position != stream.Length) {
UInt32 value = stream.ReadUInt32(position);
position += 4;
} } } }


Where BinaryReaderBigEndian class changed in:



public static class BinaryReaderBigEndian {
public override UInt32 ReadUInt32(this ReadOnlySpan<byte> stream, int start) {
var data = stream.Slice(start, 4).ToArray();
Array.Reverse(x);
return BitConverter.ToUInt32(x, 0);
} }


But, unfortunately, I didn't notice any improvement. So, where am I doing wrong?







c# html .net memorystream






share|improve this question







New contributor




heliosophist is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.











share|improve this question







New contributor




heliosophist is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









share|improve this question




share|improve this question






New contributor




heliosophist is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.









asked 2 hours ago









heliosophist

161




161




New contributor




heliosophist is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.





New contributor





heliosophist is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.






heliosophist is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.












  • where is your bottle neck? CPU, Memory, Disk access?
    – Richard Hubley
    2 hours ago










  • Nowhere! This is why I'm surprised.
    – heliosophist
    2 hours ago


















  • where is your bottle neck? CPU, Memory, Disk access?
    – Richard Hubley
    2 hours ago










  • Nowhere! This is why I'm surprised.
    – heliosophist
    2 hours ago
















where is your bottle neck? CPU, Memory, Disk access?
– Richard Hubley
2 hours ago




where is your bottle neck? CPU, Memory, Disk access?
– Richard Hubley
2 hours ago












Nowhere! This is why I'm surprised.
– heliosophist
2 hours ago




Nowhere! This is why I'm surprised.
– heliosophist
2 hours ago

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});






heliosophist is a new contributor. Be nice, and check out our Code of Conduct.










 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53416004%2fhow-to-read-a-binary-file-quickly-in-c-readonlyspan-vs-memorystream%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes








heliosophist is a new contributor. Be nice, and check out our Code of Conduct.










 

draft saved


draft discarded


















heliosophist is a new contributor. Be nice, and check out our Code of Conduct.













heliosophist is a new contributor. Be nice, and check out our Code of Conduct.












heliosophist is a new contributor. Be nice, and check out our Code of Conduct.















 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53416004%2fhow-to-read-a-binary-file-quickly-in-c-readonlyspan-vs-memorystream%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Catalogne

Violoncelliste

Héron pourpré