Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Da blob api #487

Merged
merged 9 commits into from
Nov 1, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion consensus-engine/src/types/block_id.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
/// The block id
#[derive(Clone, Copy, Debug, Default, Eq, Hash, PartialEq, Ord, PartialOrd)]

#[cfg_attr(feature = "openapi", derive(utoipa::ToSchema))]
pub struct BlockId(pub(crate) [u8; 32]);

Expand Down
24 changes: 22 additions & 2 deletions nodes/nomos-node-api/src/http/backend/axum.rs
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ use crate::{
#[derive(Clone)]
pub struct AxumBackendSettings {
pub addr: SocketAddr,
pub da_mempool: OverwatchHandle,
pub da: OverwatchHandle,
pub cl: OverwatchHandle,
pub carnot: OverwatchHandle,
Expand Down Expand Up @@ -84,6 +85,7 @@ where
.merge(SwaggerUi::new("/swagger-ui").url("/api-docs/openapi.json", ApiDoc::openapi()))
.route("/da/metrics", routing::get(da_metrics))
.route("/da/status", routing::post(da_status))
.route("/da/blob", routing::post(da_blob))
.route("/cl/metrics", routing::get(cl_metrics::<T>))
.route("/cl/status", routing::post(cl_status::<T>))
.route("/carnot/info", routing::get(carnot_info::<T, S, SIZE>))
Expand All @@ -105,7 +107,7 @@ where
)
)]
async fn da_metrics(State(store): State<Store>) -> impl IntoResponse {
match da::da_mempool_metrics(&store.da).await {
match da::da_mempool_metrics(&store.da_mempool).await {
Ok(metrics) => (StatusCode::OK, Json(metrics)).into_response(),
Err(e) => (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()).into_response(),
}
Expand All @@ -123,7 +125,25 @@ async fn da_status(
State(store): State<Store>,
Json(items): Json<Vec<<Blob as blob::Blob>::Hash>>,
) -> impl IntoResponse {
match da::da_mempool_status(&store.da, items).await {
match da::da_mempool_status(&store.da_mempool, items).await {
Ok(status) => (StatusCode::OK, Json(status)).into_response(),
Err(e) => (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()).into_response(),
}
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as in other PRs not sure why do we return IntoResponse when we already transform everything into a response inside.

Copy link
Contributor Author

@al8n al8n Oct 30, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the type returned in ok arm and err arm are not actually the same, we have to call into_response to convert to Response.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I guessed that, but why return IntoResponse later when we already got a response? Is it to avoid boxing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, we can use Response as the returned type in our case.


#[utoipa::path(
post,
path = "/da/blob",
responses(
(status = 200, description = "Query the mempool status of the da service", body = Vec<Blob>),
(status = 500, description = "Internal server error", body = String),
)
)]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we return all blobs it is better that the path is /da/blobs. Also the description would be something like Get pending blobs.

async fn da_blob(
State(store): State<Store>,
Json(items): Json<Vec<<Blob as blob::Blob>::Hash>>,
) -> impl IntoResponse {
match da::da_blob(&store.da, items).await {
Ok(status) => (StatusCode::OK, Json(status)).into_response(),
Err(e) => (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()).into_response(),
}
Expand Down
29 changes: 28 additions & 1 deletion nodes/nomos-node-api/src/http/da.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
use full_replication::{Blob, Certificate};
use full_replication::{AbsoluteNumber, Attestation, Blob, Certificate, FullReplication};
use nomos_core::da::blob;
use nomos_da::{
backend::memory_cache::BlobCache, network::adapters::libp2p::Libp2pAdapter as DaLibp2pAdapter,
DaMsg, DataAvailabilityService,
};
use nomos_mempool::{
backend::mockpool::MockPool,
network::adapters::libp2p::Libp2pAdapter,
Expand All @@ -8,13 +12,19 @@
};
use tokio::sync::oneshot;

pub(crate) type DaMempoolService = MempoolService<

Check warning on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

type alias `DaMempoolService` is never used

Check warning on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

type alias `DaMempoolService` is never used

Check warning on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Check (libp2p)

type alias `DaMempoolService` is never used

Check failure on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Rust lints (libp2p)

type alias `DaMempoolService` is never used

Check warning on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

type alias `DaMempoolService` is never used

Check warning on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

type alias `DaMempoolService` is never used

Check warning on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

type alias `DaMempoolService` is never used

Check warning on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

type alias `DaMempoolService` is never used

Check warning on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

type alias `DaMempoolService` is never used

Check warning on line 15 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

type alias `DaMempoolService` is never used
Libp2pAdapter<Certificate, <Blob as blob::Blob>::Hash>,
MockPool<Certificate, <Blob as blob::Blob>::Hash>,
CertDiscriminant,
>;

pub type DataAvailability = DataAvailabilityService<

Check warning on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

type alias `DataAvailability` is never used

Check warning on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

type alias `DataAvailability` is never used

Check warning on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Check (libp2p)

type alias `DataAvailability` is never used

Check failure on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Rust lints (libp2p)

type alias `DataAvailability` is never used

Check warning on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

type alias `DataAvailability` is never used

Check warning on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

type alias `DataAvailability` is never used

Check warning on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

type alias `DataAvailability` is never used

Check warning on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

type alias `DataAvailability` is never used

Check warning on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

type alias `DataAvailability` is never used

Check warning on line 21 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

type alias `DataAvailability` is never used
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to reuse the type in nomos-node

Copy link
Collaborator

@danielSanchezQ danielSanchezQ Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@al8n, refactor for this will be done later on? Let's open an issue for tracking it please!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, will open a new PR soon for this.

FullReplication<AbsoluteNumber<Attestation, Certificate>>,
BlobCache<<Blob as nomos_core::da::blob::Blob>::Hash, Blob>,
DaLibp2pAdapter<Blob, Attestation>,
>;

pub(crate) async fn da_mempool_metrics(

Check warning on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

function `da_mempool_metrics` is never used

Check warning on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

function `da_mempool_metrics` is never used

Check warning on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Check (libp2p)

function `da_mempool_metrics` is never used

Check failure on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Rust lints (libp2p)

function `da_mempool_metrics` is never used

Check warning on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

function `da_mempool_metrics` is never used

Check warning on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

function `da_mempool_metrics` is never used

Check warning on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

function `da_mempool_metrics` is never used

Check warning on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

function `da_mempool_metrics` is never used

Check warning on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

function `da_mempool_metrics` is never used

Check warning on line 27 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

function `da_mempool_metrics` is never used
handle: &overwatch_rs::overwatch::handle::OverwatchHandle,
) -> Result<MempoolMetrics, super::DynError> {
let relay = handle.relay::<DaMempoolService>().connect().await?;
Expand All @@ -29,7 +39,7 @@
Ok(receiver.await.unwrap())
}

pub(crate) async fn da_mempool_status(

Check warning on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

function `da_mempool_status` is never used

Check warning on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

function `da_mempool_status` is never used

Check warning on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Check (libp2p)

function `da_mempool_status` is never used

Check failure on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Rust lints (libp2p)

function `da_mempool_status` is never used

Check warning on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

function `da_mempool_status` is never used

Check warning on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

function `da_mempool_status` is never used

Check warning on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

function `da_mempool_status` is never used

Check warning on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

function `da_mempool_status` is never used

Check warning on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

function `da_mempool_status` is never used

Check warning on line 42 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

function `da_mempool_status` is never used
handle: &overwatch_rs::overwatch::handle::OverwatchHandle,
items: Vec<<Blob as blob::Blob>::Hash>,
) -> Result<Vec<Status>, super::DynError> {
Expand All @@ -45,3 +55,20 @@

Ok(receiver.await.unwrap())
}

pub(crate) async fn da_blob(

Check warning on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

function `da_blob` is never used

Check warning on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test

function `da_blob` is never used

Check warning on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Check (libp2p)

function `da_blob` is never used

Check failure on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Rust lints (libp2p)

function `da_blob` is never used

Check warning on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

function `da_blob` is never used

Check warning on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, ubuntu-latest)

function `da_blob` is never used

Check warning on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

function `da_blob` is never used

Check warning on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, windows-latest)

function `da_blob` is never used

Check warning on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

function `da_blob` is never used

Check warning on line 59 in nodes/nomos-node-api/src/http/da.rs

View workflow job for this annotation

GitHub Actions / Test Suite (libp2p, macos-latest)

function `da_blob` is never used
handle: &overwatch_rs::overwatch::handle::OverwatchHandle,
ids: Vec<<Blob as blob::Blob>::Hash>,
) -> Result<Vec<Blob>, super::DynError> {
let relay = handle.relay::<DataAvailability>().connect().await?;
let (reply_channel, receiver) = oneshot::channel();
relay
.send(DaMsg::Get {
ids: Box::new(ids.into_iter()),
reply_channel,
})
.await
.map_err(|(e, _)| e)?;

Ok(receiver.await?)
}
1 change: 1 addition & 0 deletions nomos-da/full-replication/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,7 @@ impl CertificateStrategy for AbsoluteNumber<Attestation, Certificate> {
}

#[derive(Debug, Clone, Serialize, Deserialize, Eq, Hash, PartialEq)]
#[cfg_attr(feature = "openapi", derive(utoipa::ToSchema))]

pub struct Blob {
data: Bytes,
Expand Down
Loading