Base64 and URL encoding: six mistakes and how to avoid them

Two encodings, two jobs

Base64 represents arbitrary bytes using 64 printable characters (A to Z, a to z, 0 to 9, plus and slash), with = padding at the end. It makes the data about one third larger. It is used to put binary data, such as an image or a signature, inside text formats like JSON, email or a data URL. It is not encryption: anyone can decode it.

Percent-encoding replaces characters that are not allowed in part of a URL with a percent sign and two hex digits of the UTF-8 byte, so a space becomes %20 and an accented e becomes %C3%A9. It keeps text from being mistaken for URL syntax. It does not hide anything either.

Mistake 1: using standard Base64 in a URL

The plus and slash characters, and the padding equals sign, all have meaning in URLs. Base64URL, defined in RFC 4648 section 5, swaps plus for hyphen and slash for underscore, and many systems drop the padding. JWTs use Base64URL for each of their parts. If a decoder reports an invalid character in a token, check for hyphens and underscores and switch the tool to URL-safe mode.

Mistake 2: forgetting that text has an encoding

Base64 works on bytes, not characters. To encode text you first choose a byte encoding, almost always UTF-8. The browser function btoa accepts only characters up to code point 255 and throws on anything else, such as an emoji or a Chinese character. The usual fix is to convert the string to UTF-8 bytes first, which is what this site's tool does. Decoding has the mirror problem: bytes that are not valid UTF-8 produce garbled text, so a binary file should be downloaded rather than shown.

Mistake 3: using the wrong URL function

The classic bug is encodeURI(value) on a query value. It leaves &, = and # alone, so a value such as a&b=c breaks out of its parameter. Encode each value with the component function and join the pieces yourself.

Source: MDN reference pages for encodeURIComponent and encodeURI, and the URL standard for the form encoding.

Mistake 4: encoding twice, or not at all

• Double encoding turns %20 into %2520. It appears when a library encodes a value that was already encoded. If you see %25 in a URL, someone encoded the percent sign.
• Not encoding leaves a raw # or & in a value, which truncates the URL or adds a parameter. Encode at the point where you build the URL, once.
• Decoding at the wrong layer has the opposite effect: decode a value before splitting the query string and a literal %26 turns into an & that splits the parameter.

Mistake 5: space as plus versus %20

In a path, a space must be %20. In a query string, plus is an old form convention for a space and most servers accept both. When you decode, only treat plus as a space if the data really is form encoded; otherwise a literal plus sign in a value, such as a phone number prefix, turns into a space.

Mistake 6: treating either one as security

Base64 and percent-encoding are reversible by anyone. Do not use them to hide a password, an API key or personal data in a link. If a value must not be read or altered, sign or encrypt it with a real cryptographic tool, and keep secrets out of URLs, which end up in logs and browser history.

A quick way to check

Paste the suspicious string into the Base64 tool and the URL encoder and see which one produces readable text. For a token with two dots in it, the JWT decoder splits and decodes the Base64URL parts for you. Everything runs in your browser, so tokens and URLs with private data stay on your machine.

Related

• 01Base64 Encode and Decode
• 02URL Encoder and Decoder
• 03JWT Decoder
• 04HTML Entity Encoder and Decoder