How to modify the body of a post before forwarding the request?

[gpfeifer]

Hi,
First of all: Great work! I want to modify the body (and the headers) of a post request before the request is forwarded. Any hints?
Best regards
Gregor

[danielSanchezQ]

@gpfeifer

Leveraging from the docs example, you can wrap the forwarding call in a closure for example and modify whatever you need:

use warp::Filter;
use warp::hyper::Body;
use warp_reverse_proxy::{extract_request_data_filter, proxy_to_and_forward_response, Headers};

#[tokio::main]
async fn main() {
    let hello = warp::path!("hello" / String).map(|name| format!("Hello port, {}!", name));

    // // spawn base server
    tokio::spawn(warp::serve(hello).run(([0, 0, 0, 0], 8080)));

    let request_filter = extract_request_data_filter();
    let app = warp::path!("hello")
        // build the request with data from previous filters
        .and(request_filter)
        .and_then(|path, query, method, mut headers: Headers, mut body: Body| {
            headers
                .insert("FOO_HEADER", "Foo content".parse().unwrap())
                .unwrap();
            
            proxy_to_and_forward_response(
                "127.0.0.1".to_string(),
                "".to_string(),
                path,
                query,
                method,
                headers,
                body,
            )
        });

    // spawn proxy server
    warp::serve(app).run(([0, 0, 0, 0], 3030)).await;
}

You could also just compose with map whatever you need to modify after request_filter, something like (not tested):

request_filter.map(|(param, ..., headers, body)| {
    // modify whatever here and return all elements again
})

Hope this is enough, don't doubt on contact again if you need further help :D

[danielSanchezQ]

uhm, can you share the error @rahulmr ?

[rahulmr]

I am working on windows. Please see the below ouput.

PS D:\personal\rust_apps\warp-proxy-eg> cargo -V
cargo 1.53.0 (4369396ce 2021-04-27)
PS D:\personal\rust_apps\warp-proxy-eg> rustup -V
rustup 1.24.3 (ce5817a94 2021-05-31)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.53.0 (53cb7b09b 2021-06-17)`
PS D:\personal\rust_apps\warp-proxy-eg> rustc -V
rustc 1.53.0 (53cb7b09b 2021-06-17)

Cargo.toml details

[package]
name = "warp-proxy-eg"
version = "0.1.0"
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
warp = "0.3"
warp-reverse-proxy = "0.5"
tokio = { version = "1.17.0", features = ["full"] }

Output

PS D:\personal\rust_apps\warp-proxy-eg> cargo run
   Compiling warp-proxy-eg v0.1.0 (D:\personal\rust_apps\warp-proxy-eg)
error[E0308]: mismatched types
  --> src\main.rs:28:17
   |
28 |                 body,
   |                 ^^^^ expected struct `warp::hyper::body::Bytes`, found struct `Body`

error[E0631]: type mismatch in closure arguments
  --> src\main.rs:16:10
   |
16 |         .and_then(|path, query, method, mut headers: Headers, mut body: Body| {
   |          ^^^^^^^^ ----------------------------------------------------------- found signature of `fn(FullPath, Option<String>, Method, HeaderMap, Body) -> _`
   |          |
   |          expected signature of `fn(FullPath, Option<String>, Method, HeaderMap, warp::hyper::body::Bytes) -> _`
   |
   = note: required because of the requirements on the impl of `warp::generic::Func<(FullPath, Option<String>, Method, HeaderMap, warp::hyper::body::Bytes)>` for `[closure@src\main.rs:16:19: 30:10]`

error[E0631]: type mismatch in closure arguments
  --> src\main.rs:33:17
   |
16 |         .and_then(|path, query, method, mut headers: Headers, mut body: Body| {
   |                   ----------------------------------------------------------- found signature of `fn(FullPath, Option<String>, Method, HeaderMap, Body) -> _`
...
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |                 ^^^ expected signature of `fn(FullPath, Option<String>, Method, HeaderMap, warp::hyper::body::Bytes) -> _`
   |
   = note: required because of the requirements on the impl of `warp::generic::Func<(FullPath, Option<String>, Method, HeaderMap, warp::hyper::body::Bytes)>` for `[closure@src\main.rs:16:19: 30:10]`
   = note: required because of the requirements on the impl of `warp::filter::FilterBase` for `warp::filter::and_then::AndThen<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<impl warp::Filter+Copy, 
Exact<warp::path::internal::Opaque<main::{closure#0}::__StaticPath>>>, impl warp::Filter+Copy>, impl warp::Filter+Clone>, [closure@src\main.rs:16:19: 30:10]>`

error[E0599]: the method `run` exists for struct `warp::Server<warp::filter::and_then::AndThen<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<impl warp::Filter+Copy, Exact<warp::path::internal::Opaque<main::{closure#0}::__StaticPath>>>, impl warp::Filter+Copy>, impl warp::Filter+Clone>, [closure@src\main.rs:16:19: 30:10]>>`, but its trait bounds were not satisfied
  --> src\main.rs:33:22
   |
16 |           .and_then(|path, query, method, mut headers: Headers, mut body: Body| {
   |                     -----------------------------------------------------------
   |                     |
   |                     doesn't satisfy `<_ as FnOnce<(FullPath, Option<String>, Method, HeaderMap, warp::hyper::body::Bytes)>>::Output = _`
   |                     doesn't satisfy `_: warp::generic::Func<(FullPath, Option<String>, Method, HeaderMap, warp::hyper::body::Bytes)>`
...
33 |       warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |                        ^^^ method cannot be called on `warp::Server<warp::filter::and_then::AndThen<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<impl warp::Filter+Copy, Exact<warp::path::internal::Opaque<main::{closure#0}::__StaticPath>>>, impl warp::Filter+Copy>, impl warp::Filter+Clone>, [closure@src\main.rs:16:19: 30:10]>>` due to unsatisfied trait bounds
   | 
  ::: D:\Apps\Scoop\persist\rustup\.cargo\registry\src\github.51.al-1ecc6299db9ec823\warp-0.3.2\src\filter\and_then.rs:12:1
   |
12 |   pub struct AndThen<T, F> {
   |   ------------------------ doesn't satisfy `_: warp::Filter`
...
37 | / pub struct AndThenFuture<T, F>
38 | | where
39 | |     T: Filter,
40 | |     F: Func<T::Extract>,
...  |
45 | |     state: State<T::Future, F>,
46 | | }
   | | -
   | | |
   | |_doesn't satisfy `<_ as Future>::Output = Result<_, _>`
   |   doesn't satisfy `_: Future`
   |
   = note: the following trait bounds were not satisfied:
           `warp::filter::and_then::AndThen<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<impl warp::Filter+Copy, Exact<warp::path::internal::Opaque<main::{closure#0}::__StaticPath>>>, impl warp::Filter+Copy>, impl warp::Filter+Clone>, [closure@src\main.rs:16:19: 30:10]>: warp::Filter`
           `[closure@src\main.rs:16:19: 30:10]: warp::generic::Func<(FullPath, Option<String>, Method, HeaderMap, warp::hyper::body::Bytes)>`
           `<[closure@src\main.rs:16:19: 30:10] as FnOnce<(FullPath, Option<String>, Method, HeaderMap, warp::hyper::body::Bytes)>>::Output = _`
           `<warp::filter::and_then::AndThenFuture<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<impl warp::Filter+Copy, Exact<warp::path::internal::Opaque<main::{closure#0}::__StaticPath>>>, impl 
warp::Filter+Copy>, impl warp::Filter+Clone>, [closure@src\main.rs:16:19: 30:10]> as Future>::Output = Result<_, _>`
           `warp::filter::and_then::AndThenFuture<warp::filter::and::And<warp::filter::and::And<warp::filter::and::And<impl warp::Filter+Copy, Exact<warp::path::internal::Opaque<main::{closure#0}::__StaticPath>>>, impl warp::Filter+Copy>, impl warp::Filter+Clone>, [closure@src\main.rs:16:19: 30:10]>: Future`

error[E0698]: type inside `async` block must be known in this context
  --> src\main.rs:33:28
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |                            ^ cannot infer type for type `{integer}`
   |
note: the type is part of the `async` block because of this `await`
  --> src\main.rs:33:5
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0698]: type inside `async` block must be known in this context
  --> src\main.rs:33:31
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |                               ^ cannot infer type for type `{integer}`
   |
note: the type is part of the `async` block because of this `await`
  --> src\main.rs:33:5
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0698]: type inside `async` block must be known in this context
  --> src\main.rs:33:34
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |                                  ^ cannot infer type for type `{integer}`
   |
note: the type is part of the `async` block because of this `await`
  --> src\main.rs:33:5
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0698]: type inside `async` block must be known in this context
  --> src\main.rs:33:37
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |                                     ^ cannot infer type for type `{integer}`
   |
note: the type is part of the `async` block because of this `await`
  --> src\main.rs:33:5
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0698]: type inside `async` block must be known in this context
  --> src\main.rs:33:27
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |                           ^^^^^^^^^^^^ cannot infer type for type `{integer}`
   |
note: the type is part of the `async` block because of this `await`
  --> src\main.rs:33:5
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0698]: type inside `async` block must be known in this context
  --> src\main.rs:33:41
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |                                         ^^^^ cannot infer type for type `{integer}`
   |
note: the type is part of the `async` block because of this `await`
  --> src\main.rs:33:5
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error[E0698]: type inside `async` block must be known in this context
  --> src\main.rs:33:26
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |                          ^^^^^^^^^^^^^^^^^^^^ cannot infer type for type `{integer}`
   |
note: the type is part of the `async` block because of this `await`
  --> src\main.rs:33:5
   |
33 |     warp::serve(app).run(([0, 0, 0, 0], 3040)).await;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to 11 previous errors

Some errors have detailed explanations: E0308, E0599, E0631, E0698.
For more information about an error, try `rustc --explain E0308`.
error: could not compile `warp-proxy-eg`

[rahulmr]

Thanks for helping out and giving direction.
I finally figure out how to work with this.
Body must be imported from warp_reverse_proxy previously I was trying to import if from other modules.
use warp_reverse_proxy::{extract_request_data_filter, proxy_to_and_forward_response, Headers, Body};

Now, it is working as expected without any errors.
I propose you provide such details in examples. This can be helpful for noob like me.
Generally, I see examples folder in every rust module. I have few more questions, can I open issues in here?

[rahulmr]

In short, in your example above, below changes are needed.

use warp::Filter;
use warp::hyper::Body;
use warp_reverse_proxy::{extract_request_data_filter, proxy_to_and_forward_response, Headers};

above must be changed to below

use warp::Filter;
use warp_reverse_proxy::{extract_request_data_filter, proxy_to_and_forward_response, Headers, Body};

[danielSanchezQ]

Uhm, it should be working with hyper Body. I'll take a look.
That for bubbling this up!
@rahulmr please feel free to open a PR with an example of how to handle the body and/or headers modification 😄

[gpfeifer]

Hi,
Thank you very much for your very helpful answer. I must admit that I still have some problems to understand how the magic of wrap is implemented. Your answer really helped me to understand a bit more.
Thank you!
Best regards
Gregor