This really depends on the nature of the binary data, and the constraints that “text” places on your output. First off, if your binary data is not compressed, try compressing before encoding. We can then assume that the distribution of 1/0 or individual bytes is more or less random. Now: why do you need text? … Read more
A ‘character set’ is just what it says: a properly-specified list of distinct characters. An ‘encoding’ is a mapping between a character set (typically Unicode today) and a (usually byte-based) technical representation of the characters. UTF-8 is an encoding, but not a character set. It is an encoding of the Unicode character set(*). The confusion … Read more
I’d like to answer this in a more web-like manner and in order to answer it so we need a little history. Joel Spolsky has written a very good introductionary article on the absolute minimum every dev should know on Unicode Character Encoding. Bear with me here because this is going to be somewhat of … Read more
One hex character can only represent 16 different values, i.e. 4 bits. (16 = 24) 40 × 4 = 160. And no, you need much more than 5 characters in base-36. There are totally 2160 different SHA-1 hashes. 2160 = 1640, so this is another reason why we need 40 hex digits. But 2160 = … Read more
Two years late, but here we go: The short answer is yes, unique binary/hex values will always encode to a unique base64 encoded string. BUT, multiple base64 encoded strings may represent a single binary/hex value. This is because hex bytes are not aligned with base64 ‘digits’. A single hex byte is represented by 8 bits … Read more
As suggested here: Get-Content tmp.log | Out-File -Encoding UTF8 new.log
iconv -f WINDOWS-1252 -t UTF-8 filename.txt
ASCII is fundamental Originally 1 character was always stored as 1 byte. A byte (8 bits) has the potential to distinct 256 possible values. But in fact only the first 7 bits were used. So only 128 characters were defined. This set is known as the ASCII character set. 0x00 – 0x1F contain steering codes … Read more
Simple enable ssl proxying on right click for specific url:
Serializing is about moving structured data over a storage/transmission medium in a way that the structure can be maintained. Encoding is more broad, like about how said data is converted to different forms, etc. Perhaps you could think about serializing being a subset of encoding in this example. With regard to a web service, you … Read more