FastHTTP parsing is messing up my request

[ruizlenato]

I need to make a GET request using fashttp but my URL is being parsed and I can't get the correct response.
I've added DisablePathNormalizing: true to my client but it's still parsing.

Original URL: http://sns-webpic-qc.xhscdn.com/202501220243/0cefc8f02aa7fbfa04da7dcd33cb0c16/notes_pre_post/1040g3k831c60utqsgmag49snd3drpkggob6652o!nd_dft_wlteh_jpg_3
Parsed: http://sns-webpic-qc.xhscdn.com/202501220243/0cefc8f02aa7fbfa04da7dcd33cb0c16/notes_pre_post/1040g3k831c60utqsgmag49snd3drpkggob6652o%21nd_dft_wlteh_jpg_3

The parsed URL doesn't work, here's an example in my browser.
Original:

Parsed:

[erikdubbelboer]

Try setting Request.DisableRedirectPathNormalizing to true.

[ruizlenato]

Try setting Request.DisableRedirectPathNormalizing to true.

I tried it earlier today and it's still the same thing.

[erikdubbelboer]

Can you share a reproducible example?

[ruizlenato]

Can you share a reproducible example?

Without Request.DisableRedirectPathNormalizing:

➜ cat main.go
package main

import (
        "fmt"

        "github.com/valyala/fasthttp"
)

const url = "http://sns-webpic-qc.xhscdn.com/202501220243/0cefc8f02aa7fbfa04da7dcd33cb0c16/notes_pre_post/1040g3k831c60utqsgmag49snd3drpkggob6652o!nd_dft_wlteh_jpg_3"

func main() {
        client := &fasthttp.Client{}

        request := fasthttp.AcquireRequest()
        response := fasthttp.AcquireResponse()

        defer fasthttp.ReleaseRequest(request)
        defer fasthttp.ReleaseResponse(response)

        request.SetRequestURI(url)

        if err := client.Do(request, response); err != nil {
                fmt.Println("Error:", err)
        }

        fmt.Println("Status code:", response.StatusCode())
        fmt.Println("Request URL", request.URI().String())
}
➜ go run main.go
Status code: 403
Request URL http://sns-webpic-qc.xhscdn.com/202501220243/0cefc8f02aa7fbfa04da7dcd33cb0c16/notes_pre_post/1040g3k831c60utqsgmag49snd3drpkggob6652o%21nd_dft_wlteh_jpg_3

With Request.DisableRedirectPathNormalizing:

➜ cat main.go
package main

import (
        "fmt"

        "github.com/valyala/fasthttp"
)

const url = "http://sns-webpic-qc.xhscdn.com/202501220243/0cefc8f02aa7fbfa04da7dcd33cb0c16/notes_pre_post/1040g3k831c60utqsgmag49snd3drpkggob6652o!nd_dft_wlteh_jpg_3"

func main() {
        client := &fasthttp.Client{}

        request := fasthttp.AcquireRequest()
        response := fasthttp.AcquireResponse()

        defer fasthttp.ReleaseRequest(request)
        defer fasthttp.ReleaseResponse(response)

        request.SetRequestURI(url)
        request.DisableRedirectPathNormalizing = true

        if err := client.Do(request, response); err != nil {
                fmt.Println("Error:", err)
        }

        fmt.Println("Status code:", response.StatusCode())
        fmt.Println("Request URL", request.URI().String())
}
➜ go run main.go
Status code: 403
Request URL http://sns-webpic-qc.xhscdn.com/202501220243/0cefc8f02aa7fbfa04da7dcd33cb0c16/notes_pre_post/1040g3k831c60utqsgmag49snd3drpkggob6652o%21nd_dft_wlteh_jpg_3

[erikdubbelboer]

Sorry by bad, you have to set client.DisablePathNormalizing = true before you make the request. We have the DisablePathNormalizing at way to many levels and I agree it's super confusing.

[ruizlenato]

Sorry by bad, you have to set client.DisablePathNormalizing = true before you make the request. We have the DisablePathNormalizing at way to many levels and I agree it's super confusing.

and when I use DoRedirects?

➜ bat main.go
───────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: main.go
───────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1   │ package main
   2   │
   3   │ import (
   4   │     "fmt"
   5   │
   6   │     "github.com/valyala/fasthttp"
   7   │ )
   8   │
   9   │ const url = "http://sns-webpic-qc.xhscdn.com/202501220243/0cefc8f02aa7fbfa04da7dcd33cb0c16/notes_pre_post/1040g3k831c60utqsgmag49snd3drpkggob6652o!nd_dft_wlteh_jpg_3"
  10   │
  11   │ func main() {
  12   │     client := &fasthttp.Client{}
  13   │
  14   │     request := fasthttp.AcquireRequest()
  15   │     response := fasthttp.AcquireResponse()
  16   │
  17   │     defer fasthttp.ReleaseRequest(request)
  18   │     defer fasthttp.ReleaseResponse(response)
  19   │
  20   │     client.DisablePathNormalizing = true
  21   │     request.SetRequestURI(url)
  22   │
  23   │     if err := client.DoRedirects(request, response, 2); err != nil {
  24   │         fmt.Println("Error:", err)
  25   │     }
  26   │
  27   │     fmt.Println("Status code:", response.StatusCode())
  28   │     fmt.Println("Request URL", request.URI().String())
  29   │ }
───────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
➜ go run main.go
Status code: 403
Request URL http://sns-webpic-qc.xhscdn.com/202501220243/0cefc8f02aa7fbfa04da7dcd33cb0c16/notes_pre_post/1040g3k831c60utqsgmag49snd3drpkggob6652o%21nd_dft_wlteh_jpg_3

[erikdubbelboer]

I can't test cause I always seem to get a 403 on that url, also when I request it with curl or a browser.