Base64 encoding/decoding will have unsafe url characters. Generally the best solution to this problem is url_encode/url_decode the base64 encoded string (most languages have simple method for url_encoding/decoding): what if we keep going so that there could be some overlap if we are not.
Another approach is to replace the url unsafe character when encoding then replace them back when decoding like so:
Note: Most languages have some built in function for url safe base64 encoding/decoding.