Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix tests that sometimes fail #359

Closed
sebastian-peter opened this issue Sep 29, 2022 · 2 comments · Fixed by #484
Closed

Fix tests that sometimes fail #359

sebastian-peter opened this issue Sep 29, 2022 · 2 comments · Fixed by #484
Assignees
@sebastian-peter
Labels
ci Related to continuous integration test Issues related to (unit) tests

Comments

@sebastian-peter
Copy link
Member

Some tests usually succeed, but sometimes don't (e.g. under high cpu load, some timeouts are set too low).
@sebastian-peter sebastian-peter added ci Related to continuous integration test Issues related to (unit) tests labels Sep 29, 2022
@sebastian-peter sebastian-peter self-assigned this Sep 29, 2022
@sebastian-peter
Copy link
Member Author

sebastian-peter commented Sep 29, 2022 •
edited
Loading

Examples:

(this first one happens a lot)

- must return valid log messages for each status event *** FAILED *** (10 seconds, 354 milliseconds)
  java.lang.AssertionError: Timeout (10 seconds) waiting for 1 messages on LoggingTestKitImpl(1,Some(INFO),None,None,Some(Simulating from 2011-01-01 00:00:00 until 2011-01-01 01:00:00.),None,None,Map(),true,None).
  at akka.actor.testkit.typed.internal.LoggingTestKitImpl.expect(LoggingTestKitImpl.scala:103)
  at edu.ie3.simona.event.listener.RuntimeEventListenerSpec.$anonfun$new$5(RuntimeEventListenerSpec.scala:155)
  at edu.ie3.simona.event.listener.RuntimeEventListenerSpec.$anonfun$new$5$adapted(RuntimeEventListenerSpec.scala:151)
  at scala.collection.immutable.List.foreach(List.scala:333)
  at edu.ie3.simona.event.listener.RuntimeEventListenerSpec.$anonfun$new$4(RuntimeEventListenerSpec.scala:151)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  ...

- should start the simulation, do 2 sweeps and should end afterwards when no deviation on nodal power is recognized in the superior when a StartGridSimulationTrigger is send *** FAILED *** (3 seconds, 875 milliseconds)
  java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg:
  at scala.Predef$.assert(Predef.scala:279)
  at akka.testkit.TestKitBase.expectMsgPF(TestKit.scala:479)
  at akka.testkit.TestKitBase.expectMsgPF$(TestKit.scala:476)
  at akka.testkit.TestKit.expectMsgPF(TestKit.scala:973)
  at edu.ie3.simona.agent.grid.DBFSAlgorithmSupGridSpec.$anonfun$new$6(DBFSAlgorithmSupGridSpec.scala:209)
  at edu.ie3.simona.agent.grid.DBFSAlgorithmSupGridSpec.$anonfun$new$6$adapted(DBFSAlgorithmSupGridSpec.scala:163)
  at scala.collection.immutable.Range.foreach(Range.scala:190)
  at edu.ie3.simona.agent.grid.DBFSAlgorithmSupGridSpec.$anonfun$new$5(DBFSAlgorithmSupGridSpec.scala:163)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  ...

- should start the simulation when a StartGridSimulationTrigger is sent, do 5 sweeps and should end afterwards, if the nodal power exchange converges not before the fifth sweep. *** FAILED *** (1 second, 168 milliseconds)
  java.lang.AssertionError: assertion failed: expected CompletionMessage(1,Some(List(ScheduleTriggerMessage(StartGridSimulationTrigger(3600),Actor[akka://DBFSAlgorithmSpec/user/$a#1627353582])))), found CompletionMessage(3,Some(Vector(ScheduleTriggerMessage(ActivityStartTrigger(7200),Actor[akka://DBFSAlgorithmSpec/user/$a#1627353582]))))
  at scala.Predef$.assert(Predef.scala:279)
  at akka.testkit.TestKitBase.expectMsg_internal(TestKit.scala:462)
  at akka.testkit.TestKitBase.expectMsg(TestKit.scala:438)
  at akka.testkit.TestKitBase.expectMsg$(TestKit.scala:438)
  at akka.testkit.TestKit.expectMsg(TestKit.scala:973)
  at edu.ie3.simona.agent.grid.DBFSAlgorithmSupGridSpec.$anonfun$new$8(DBFSAlgorithmSupGridSpec.scala:307)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  ...

https://simona.ie3.e-technik.tu-dortmund.de/ci/job/ie3-institute/job/simona/job/dev/268/console

- should start the simulation when a StartGridSimulationTrigger is sent, handle failed power flow if it occurs *** FAILED *** (3 seconds, 146 milliseconds)
  java.lang.AssertionError: assertion failed: timeout (3 seconds) during expectMsg while waiting for FailedPowerFlow
  at scala.Predef$.assert(Predef.scala:279)
  at akka.testkit.TestKitBase.expectMsg_internal(TestKit.scala:461)
  at akka.testkit.TestKitBase.expectMsg(TestKit.scala:438)
  at akka.testkit.TestKitBase.expectMsg$(TestKit.scala:438)
  at akka.testkit.TestKit.expectMsg(TestKit.scala:973)
  at edu.ie3.simona.agent.grid.DBFSAlgorithmFailedPowerFlowSpec.$anonfun$new$5(DBFSAlgorithmFailedPowerFlowSpec.scala:242)
  at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  ...

https://simona.ie3.e-technik.tu-dortmund.de/ci/job/ie3-institute/job/simona/job/dev/270/console

- should write a series of new NodeResults to Kafka *** FAILED *** (22 seconds, 213 milliseconds)
  The code passed to eventually never returned normally. Attempted 13 times over 21.375327958000003 seconds. Last failure message: List() had length 0 instead of expected length 3. (ResultEntityKafkaSpec.scala:156)
  org.scalatest.exceptions.TestFailedDueToTimeoutException:
  ...
  at edu.ie3.simona.io.result.ResultEntityKafkaSpec.$anonfun$new$2(ResultEntityKafkaSpec.scala:156)
  at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  at org.scalatest.wordspec.AnyWordSpecLike$$anon$3.apply(AnyWordSpecLike.scala:1239)
  ...
  Cause: org.scalatest.exceptions.TestFailedException: List() had length 0 instead of expected length 3
  ...
  at edu.ie3.simona.io.result.ResultEntityKafkaSpec.$anonfun$new$3(ResultEntityKafkaSpec.scala:160)
  at org.scalatest.enablers.Retrying$$anon$4.makeAValiantAttempt$1(Retrying.scala:184)
  at org.scalatest.enablers.Retrying$$anon$4.tryTryAgain$2(Retrying.scala:196)
  at org.scalatest.enablers.Retrying$$anon$4.retry(Retrying.scala:226)
  at org.scalatest.concurrent.Eventually.eventually(Eventually.scala:313)
  at org.scalatest.concurrent.Eventually.eventually$(Eventually.scala:312)
  at edu.ie3.simona.io.result.ResultEntityKafkaSpec.eventually(ResultEntityKafkaSpec.scala:43)

https://simona.ie3.e-technik.tu-dortmund.de/ci/job/ie3-institute/job/simona/job/dev/267/console

  - should shutdown and compress the data when requested to do so without any errors *** FAILED *** (10 seconds, 386 milliseconds)
    java.lang.AssertionError: assertion failed: timeout 10 seconds expired:
    at scala.Predef$.assert(Predef.scala:279)
    at akka.testkit.TestKitBase.poll$1(TestKit.scala:300)
    at akka.testkit.TestKitBase.awaitCond(TestKit.scala:306)
    at akka.testkit.TestKitBase.awaitCond$(TestKit.scala:289)
    at akka.testkit.TestKit.awaitCond(TestKit.scala:973)
    at edu.ie3.simona.event.listener.ResultEventListenerSpec.$anonfun$new$37(ResultEventListenerSpec.scala:563)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
    at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
    at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
    at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
    ...

@sebastian-peter sebastian-peter removed their assignment Oct 7, 2022
@staudtMarius
Copy link
Member

staudtMarius commented Feb 17, 2023 •
edited by sebastian-peter
Loading

I found another one.

- should process a valid participants result correctly *** FAILED *** (10 seconds, 23 milliseconds)
      java.lang.AssertionError: assertion failed: timeout 10 seconds expired:
      at scala.Predef$.assert(Predef.scala:279)
      at akka.testkit.TestKitBase.poll$1(TestKit.scala:300)
      at akka.testkit.TestKitBase.awaitCond(TestKit.scala:306)
      at akka.testkit.TestKitBase.awaitCond$(TestKit.scala:289)
      at akka.testkit.TestKit.awaitCond(TestKit.scala:973)
      at edu.ie3.simona.event.listener.ResultEventListenerSpec.$anonfun$new$7(ResultEventListenerSpec.scala:192)  
      at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18)
      at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85)
      at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83)
      at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
      ...

https://simona.ie3.e-technik.tu-dortmund.de/ci/job/ie3-institute/job/simona/job/dev/272/console

@sebastian-peter sebastian-peter self-assigned this Mar 10, 2023
@sebastian-peter sebastian-peter mentioned this issue Mar 10, 2023
Merged
danielfeismann added a commit that referenced this issue Mar 12, 2023
@danielfeismann
Merge pull request #484 from ie3-institute/sp/#359-fix-unreliable-tests
3c978cb 
Fix unreliable tests that sometimes fail (with timeouts)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sebastian-peter sebastian-peter

Labels
ci Related to continuous integration test Issues related to (unit) tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix unreliable tests that sometimes fail (with timeouts) ie3-institute/simona
2 participants
@sebastian-peter @staudtMarius