Using deck with Cassandra datastore causes 409 errors #154
Comments
|
We’ve also run into this issue and have mitigated it with the —parallelism 1 switch. |
|
We already tested the parallelism to to 1 but with a big number of changes the 409 error appeared in a intermittent way. |
|
Tracking the fix for this in #160 to fix this issue. |
hbagdi
added a commit
that referenced
this issue
Jun 26, 2020
decK first creates all the services and then creates related routes. With a distributed like Cassandra, route creation happens before the DB rows for new services are propagated to other nodes in the cassandra cluster. This results in a 409 from kong because kong ensures that service specified in a route is actually present in the db (c* has no notion of foreign relations, kong does a read on the service to verify and enforce foreign key relations). With the new configurable flag, users of cassandra can inject an aritifical between insert operations of related entities to avoid failures described above. The use of the flag is discouraged and is only implemented to provide a stop-gap solution. Fix #160 Fix #154
hbagdi
added a commit
that referenced
this issue
Jun 26, 2020
decK first creates all the services and then creates related routes. With a distributed like Cassandra, route creation happens before the DB rows for new services are propagated to other nodes in the cassandra cluster. This results in a 409 from kong because kong ensures that service specified in a route is actually present in the db (c* has no notion of foreign relations, kong does a read on the service to verify and enforce foreign key relations). With the new configurable flag, users of cassandra can inject an aritifical between insert operations of related entities to avoid failures described above. The use of the flag is discouraged and is only implemented to provide a stop-gap solution. Fix #160 Fix #154
hbagdi
added a commit
that referenced
this issue
Jun 26, 2020
decK first creates all the services and then creates related routes. With a distributed like Cassandra, route creation happens before the DB rows for new services are propagated to other nodes in the cassandra cluster. This results in a 409 from kong because kong ensures that service specified in a route is actually present in the db (c* has no notion of foreign relations, kong does a read on the service to verify and enforce foreign key relations). With the new configurable flag, users of cassandra can inject an aritifical between insert operations of related entities to avoid failures described above. The use of the flag is discouraged and is only implemented to provide a stop-gap solution. Fix #160 Fix #154 From #179
rainest
pushed a commit
that referenced
this issue
Apr 21, 2021
decK first creates all the services and then creates related routes. With a distributed like Cassandra, route creation happens before the DB rows for new services are propagated to other nodes in the cassandra cluster. This results in a 409 from kong because kong ensures that service specified in a route is actually present in the db (c* has no notion of foreign relations, kong does a read on the service to verify and enforce foreign key relations). With the new configurable flag, users of cassandra can inject an aritifical between insert operations of related entities to avoid failures described above. The use of the flag is discouraged and is only implemented to provide a stop-gap solution. Fix #160 Fix #154
rainest
pushed a commit
that referenced
this issue
Apr 21, 2021
decK first creates all the services and then creates related routes. With a distributed like Cassandra, route creation happens before the DB rows for new services are propagated to other nodes in the cassandra cluster. This results in a 409 from kong because kong ensures that service specified in a route is actually present in the db (c* has no notion of foreign relations, kong does a read on the service to verify and enforce foreign key relations). With the new configurable flag, users of cassandra can inject an aritifical between insert operations of related entities to avoid failures described above. The use of the flag is discouraged and is only implemented to provide a stop-gap solution. Fix #160 Fix #154 From #179
AntoineJac
pushed a commit
that referenced
this issue
Jan 23, 2024
decK first creates all the services and then creates related routes. With a distributed like Cassandra, route creation happens before the DB rows for new services are propagated to other nodes in the cassandra cluster. This results in a 409 from kong because kong ensures that service specified in a route is actually present in the db (c* has no notion of foreign relations, kong does a read on the service to verify and enforce foreign key relations). With the new configurable flag, users of cassandra can inject an aritifical between insert operations of related entities to avoid failures described above. The use of the flag is discouraged and is only implemented to provide a stop-gap solution. Fix #160 Fix #154
AntoineJac
pushed a commit
that referenced
this issue
Jan 23, 2024
decK first creates all the services and then creates related routes. With a distributed like Cassandra, route creation happens before the DB rows for new services are propagated to other nodes in the cassandra cluster. This results in a 409 from kong because kong ensures that service specified in a route is actually present in the db (c* has no notion of foreign relations, kong does a read on the service to verify and enforce foreign key relations). With the new configurable flag, users of cassandra can inject an aritifical between insert operations of related entities to avoid failures described above. The use of the flag is discouraged and is only implemented to provide a stop-gap solution. Fix #160 Fix #154 From #179
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
deck runs Admin API calls too fast for Cassandra to propagate changes to all nodes. This causes intermittent 409 errors when the Cassandra nodes do not have the data yet for subsequent API calls. Request to add a configurable delay to deck, similar to
db_update_propagation, that will allow Cassandra to catchupThe text was updated successfully, but these errors were encountered: