Add sync fitting #773

RayCiel · 2021-04-30T06:36:41Z

Signed off.

vmwclabot · 2021-04-30T06:36:44Z

@RayCiel, you must sign every commit in this pull request acknowledging our Developer Certificate of Origin before your changes are merged. This can be done by adding Signed-off-by: John Doe <john.doe@email.org> to the last line of each Git commit message. The e-mail address used to sign must match the e-mail address of the Git author. Click here to view the Developer Certificate of Origin agreement.

Commit f028eaf must be signed by loul@vmware.com
Commit 78cbe99 must be signed by loul@vmware.com
Commit 5b5788c must be signed by loul@vmware.com
Commit 2bbf930 must be signed by loul@vmware.com
Commit f085412 must be signed by loul@vmware.com
Commit e8c71c7 must be signed by loul@vmware.com

delete error/logs fix bugs fix bugs add test & fix bugs delete useless logs signed off by Signed-off-by: Lan Lou <loul@vmware.com> merge commit and signed off Signed-off-by: Lan Lou <loul@vmware.com> add sync fitting delete error/logs fix bugs fix bugs add test & fix bugs delete useless logs signed off by Signed-off-by: Lan Lou <loul@vmware.com>

Signed-off-by: Lan Lou <loul@vmware.com>

yixingjia

please don't commit binary files

yixingjia · 2021-05-13T12:19:04Z

aggregator/src/main/java/com/vmware/flowgate/aggregator/config/ServiceKeyConfig.java

@@ -18,6 +18,9 @@ public ServiceKeyConfig() {
   }

   public String getServiceKey() {
-      return serviceKey;


please check those code before create the pr.

yixingjia · 2021-05-13T12:20:06Z

aggregator/src/main/java/com/vmware/flowgate/aggregator/scheduler/job/AggregatorService.java

@@ -65,10 +77,10 @@ public void executeAsync(EventMessage message) {
         logger.warn("Drop none aggregator message " + message.getType());
         return;
      }
-      logger.info(message.getContent());
+      //logger.info("message: " + message.getContent());


? why remove the log info?

yixingjia · 2021-05-13T12:21:12Z

aggregator/src/main/java/com/vmware/flowgate/aggregator/scheduler/job/AggregatorService.java

@@ -92,6 +104,19 @@ public void executeAsync(EventMessage message) {
         case EventMessageUtil.CleanRealtimeData:
            cleanRealtimeData();
            break;
+         case EventMessageUtil.SYNC_FITTING:
+        	List<List<Double>> results = syncFitting(false);


yixingjia · 2021-05-13T12:21:53Z

aggregator/src/main/java/com/vmware/flowgate/aggregator/scheduler/job/AggregatorService.java

@@ -103,7 +128,247 @@ public void executeAsync(EventMessage message) {
         }
      }
   }
+
+   public List<Double> doubleToList(double[] arr_double) {


can you move those math related code into a separate file?

yixingjia · 2021-05-13T12:22:49Z

aggregator/src/main/java/com/vmware/flowgate/aggregator/scheduler/job/AggregatorService.java

+		  try {
+			  	 MetricDatas = new ArrayList<>();
+
+		         CsvReader csvReader = new CsvReader("testData.csv");


you should change to api for the when you submit pr

yixingjia · 2021-05-13T12:23:50Z

aggregator/src/main/resources/logback-spring.xml

@@ -7,10 +7,10 @@
    <include resource="org/springframework/boot/logging/logback/base.xml" />  
 	<appender name="fileInfoLog"
 		class="ch.qos.logback.core.rolling.RollingFileAppender">
-		<File>/log/aggregator-info.log</File>
+		<File>aggregator-info.log</File>


change back the settings, please don't submit code which only works in your env.

yixingjia · 2021-05-13T12:24:33Z

aggregator/src/test/java/com/vmware/flowgate/aggregator/MessageProcessingTest.java

@@ -494,26 +496,6 @@ public void testOpenManageJobExecute1() throws JobExecutionException, JsonProces
      openmanageJob.execute(null);
   }



why remove those code ?

Some code submitted after I first pull the flowgate. I will check and add them.

yixingjia · 2021-05-13T12:25:29Z

flowgate-api/pom.xml

@@ -30,9 +30,8 @@
 		<jwt.version>3.3.0</jwt.version>
 		<commons-io.version>2.4</commons-io.version>
 		<jackson.version>2.10.0</jackson.version>
-		<tomcat.version>9.0.41</tomcat.version>


change back.

yixingjia · 2021-05-13T12:25:54Z

flowgate-api/pom.xml

 		<maven-jar-plugin.version>3.1.1</maven-jar-plugin.version>
-		<guava.version>30.0-jre</guava.version>


don't overwrite other commit.

yixingjia · 2021-05-13T12:26:29Z

flowgate-api/pom.xml

@@ -112,7 +111,7 @@
 		<dependency>
 			<groupId>com.google.guava</groupId>
 			<artifactId>guava</artifactId>
-			<version>${guava.version}</version>
+			<version>28.0-jre</version>


? why we change from parameter to a hardcode value?

yixingjia · 2021-05-13T12:26:46Z

flowgate-api/pom.xml

@@ -183,11 +182,6 @@
 			<artifactId>woodstox-core</artifactId>
 			<version>5.0.2</version>
 		</dependency>
-		<dependency>


why do we remove this?

yixingjia · 2021-05-13T12:27:07Z

flowgate-api/src/main/java/com/vmware/flowgate/config/InitializeConfigureData.java

@@ -60,7 +60,8 @@ public static void init(
      if(!isInitialized) {
         InitializeConfigureData.roleNameAndPrivilegeMap = roleNameAndPrivilegeMap;
         InitializeConfigureData.privilegeResourceMap = privilegeResourceMap;
-         serviceKey = System.getenv(FlowgateConstant.serviceKey);
+         serviceKey = "loul"; 


remove the any hacks before submit code.

yixingjia · 2021-05-13T12:28:11Z

flowgate-api/src/main/java/com/vmware/flowgate/service/AssetService.java

@@ -13,13 +13,11 @@
 import java.util.Collections;
 import java.util.HashMap;
 import java.util.HashSet;
-import java.util.Iterator;


Double check this file, I believe most of the code are delete by mistake.

yixingjia · 2021-05-13T12:28:43Z

flowgate-api/src/main/resources/application.properties

@@ -23,7 +23,7 @@ logging.config=classpath:logback-spring.xml
 #######redis setting#######
 spring.redis.host=localhost
 spring.redis.port=6379
-spring.redis.password=REDISPASSWD_CHANGE
+#spring.redis.password=REDISPASSWD_CHANGE


change back all the hackings!

yixingjia · 2021-05-13T12:28:59Z

flowgate-api/src/main/resources/logback-spring.xml

@@ -21,7 +20,7 @@

 	<appender name="fileErrorLog"
 		class="ch.qos.logback.core.rolling.RollingFileAppender">
-		<File>/log/flowgate-api-error.log</File>
+		<File>flowgate-api-error.log</File>


change back all the hackings!

yixingjia · 2021-05-13T12:29:28Z

flowgate-api/src/test/java/com/vmware/flowgate/controller/AssetControllerTest.java

@@ -64,6 +64,7 @@
 import com.vmware.flowgate.common.MetricKeyName;
 import com.vmware.flowgate.common.MetricName;
 import com.vmware.flowgate.common.MountingSide;
+import com.vmware.flowgate.common.RealtimeDataUnit;


double check this file.

yixingjia · 2021-05-13T12:30:02Z

flowgate-api/src/test/resources/application.properties

@@ -5,7 +5,7 @@ server.port=49610

 ########couchbase config######
 spring.couchbase.env.timeouts.query=30000
-spring.couchbase.bootstrap-hosts=localhost
+spring.couchbase.bootstrap-hosts=database-build


change back.

yixingjia · 2021-05-13T12:30:21Z

flowgate-api/src/test/resources/application.properties-e

@@ -0,0 +1,22 @@
+#Copyright 2019 VMware, Inc.


why do we need this file?

yixingjia · 2021-05-13T12:30:31Z

flowgate-common/src/main/java/com/vmware/flowgate/common/FlowgateConstant.java

@@ -12,7 +12,7 @@

   public static int defaultPageNumber = 1;

-   public static String serviceKey = "FLOWGATESERVICEKEY";
+   public static String serviceKey = "serviceKey";//"FLOWGATESERVICEKEY";


change back.

yixingjia · 2021-05-13T12:31:11Z

flowgate-common/src/main/java/com/vmware/flowgate/common/model/metadata/PredefinedSDDCJobs.java

@@ -13,7 +13,7 @@
 public class PredefinedSDDCJobs {
   public static final List<JobConfig> ALLJobs = Arrays.asList(
         new JobConfig("AGGREGATOR-PREDEFINED-AGGREGATOR-JOB-DISPATCHER", "AggregateJobDispatcher", "AggregateJobs",
-               "Hourly", "AggregatorJob", "", "0 23 * * * ?",
+               "FiveMinutes", "AggregatorJob", "", "0 0/5 * * * ?",


why do we change this part?

Signed-off-by: Lan Lou <loul@vmware.com>

vmwclabot added the dco-required label Apr 30, 2021

RayCiel force-pushed the master branch from a2cf5b2 to 8eecc6d Compare April 30, 2021 07:21

vmwclabot removed the dco-required label Apr 30, 2021

Lan Lou added 2 commits April 30, 2021 15:28

merge commit and signed off

67e2847

Signed-off-by: Lan Lou <loul@vmware.com>

modify

b00dc2b

Signed-off-by: Lan Lou <loul@vmware.com>

yixingjia reviewed May 13, 2021

View reviewed changes

Lan Lou added 2 commits May 19, 2021 11:40

merge & remove hackings

94313ad

Signed-off-by: Lan Lou <loul@vmware.com>

optimize

d0e4d9b

Signed-off-by: Lan Lou <loul@vmware.com>

Lan Lou added 3 commits July 8, 2021 18:04

add redeploy recommend

ce2475c

Signed-off-by: Lan Lou <loul@vmware.com>

fix bugs

730fb73

Signed-off-by: Lan Lou <loul@vmware.com>

delete useless annotations

dec9c8c

Signed-off-by: Lan Lou <loul@vmware.com>

		@@ -494,26 +496,6 @@ public void testOpenManageJobExecute1() throws JobExecutionException, JsonProces
		openmanageJob.execute(null);
		}

		<maven-jar-plugin.version>3.1.1</maven-jar-plugin.version>
		<guava.version>30.0-jre</guava.version>

Add sync fitting #773

Are you sure you want to change the base?

Add sync fitting #773

Uh oh!

Conversation

RayCiel commented Apr 30, 2021

Uh oh!

vmwclabot commented Apr 30, 2021

Uh oh!

yixingjia left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!