Encoding error in some URLs #10

New Issue

Open

opened 2022-09-22 05:40:43 -05:00 by Ambrevar · 1 comment

Ambrevar commented

2022-09-22 05:40:43 -05:00

(Migrated from github.com)

I've spotted at least 2 bugs:

url-encode asks for utf-8 encoding, but it seems wrong to change it from the default. In particular puni-encoded URLs won't like this. You can test the problem with bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq/I/m/Van_Gogh_-_Bildnis_der_Mutter_des_K%C3%BCnstlers.jpeg. Fix:

(defun url-encode (string &rest ignored)
  "Wrap around drakma's url encoder, with a slight change-- instead of using
  plus-signs for spaces, we want to use %20."
  ignored
  (cl-ppcre:regex-replace-all
   "%2520" (drakma:url-encode
            (cl-ppcre:regex-replace-all " " string "%20")
            drakma:*drakma-default-external-format*)
   "%20"))

Subtle one, but the URL in ipfs-call on SBCL is converted from an (array character) to a (array base-char), which breaks puri on URL with non-ascii characters. Puri's fault, but you can implement a local fix

(drakma:http-request
        (let ((url (make-call-url call arguments)))
          (make-array (array-dimensions url)
                      :initial-contents url
                      :element-type 'character))
        :method method
        :url-encoder #'ipfs::url-encode
        :parameters parameters
        :want-stream want-stream)

A dedicated function or even the use of alexandria:copy-array would be a better fix, but I'm keeping this as short as possible for this thread.

I've spotted at least 2 bugs: 1. `url-encode` asks for `utf-8` encoding, but it seems wrong to change it from the default. In particular puni-encoded URLs won't like this. You can test the problem with `bafybeiemxf5abjwjbikoz4mc3a3dla6ual3jsgpdr4cjr3oz3evfyavhwq/I/m/Van_Gogh_-_Bildnis_der_Mutter_des_K%C3%BCnstlers.jpeg`. Fix: ```lisp (defun url-encode (string &rest ignored) "Wrap around drakma's url encoder, with a slight change-- instead of using plus-signs for spaces, we want to use %20." ignored (cl-ppcre:regex-replace-all "%2520" (drakma:url-encode (cl-ppcre:regex-replace-all " " string "%20") drakma:*drakma-default-external-format*) "%20")) ``` 2. Subtle one, but the URL in `ipfs-call` on SBCL is converted from an `(array character)` to a `(array base-char)`, which breaks puri on URL with non-ascii characters. Puri's fault, but you can implement a local fix ```lisp (drakma:http-request (let ((url (make-call-url call arguments))) (make-array (array-dimensions url) :initial-contents url :element-type 'character)) :method method :url-encoder #'ipfs::url-encode :parameters parameters :want-stream want-stream) ``` A dedicated function or even the use of `alexandria:copy-array` would be a better fix, but I'm keeping this as short as possible for this thread.

JadedCtrl commented

2022-10-06 20:31:19 -05:00

(Migrated from github.com)

Would you care to commit this patch on a branch of your clone? I could apply it locally, but you wouldn't get the credit.