Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

label updater giving exception #580

Closed
vikasgupta78 opened this issue May 8, 2023 · 4 comments
Closed

label updater giving exception #580

vikasgupta78 opened this issue May 8, 2023 · 4 comments
Milestone
0.3.5

Comments

@vikasgupta78
Copy link
Contributor

vikasgupta78 commented May 8, 2023 •
edited
Loading

When trying to update label for a cluster currently marked not sure, exception occurs.

Steps followed:

  1. ./scripts/zingg.sh --phase findTrainingData --conf examples/febrl/config.json --zinggDir /tmp/z_lblupd

  2. ./scripts/zingg.sh --phase label --conf examples/febrl/config.json --zinggDir /tmp/z_lblupd

  3. ./scripts/zingg.sh --phase trainMatch --conf examples/febrl/config.json --zinggDir /tmp/z_lblupd

  4. ./scripts/zingg.sh --phase generateDocs --conf examples/febrl/config.json --zinggDir /tmp/z_lblupd

(genertaed docs attached as pdf)

  1. In generated docs 1683506973234:7 was present as not sure

  2. ./scripts/zingg.sh --phase updateLabel --conf examples/febrl/config.json --zinggDir /tmp/z_lblupd

    Labelled pairs so far : 8/18 MATCH, 8/18 DO NOT MATCH, 2/18 NOT SURE

    Please enter the cluster id (or 9 to exit): 1683506973234:7
    java.lang.NullPointerException
    at zingg.common.core.executor.LabelUpdater.processRecordsCli(LabelUpdater.java:67)
    at zingg.common.core.executor.LabelUpdater.execute(LabelUpdater.java:28)
    at zingg.common.client.Client.execute(Client.java:243)
    at zingg.common.client.Client.mainMethod(Client.java:182)
    at zingg.spark.client.SparkClient.main(SparkClient.java:65)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
    2023-05-08 06:26:47,349 [main] WARN zingg.common.core.executor.LabelUpdater - An error has occured while Updating Label. null
    2023-05-08 06:26:49,669 [main] WARN zingg.common.client.util.Email - Unable to send email Can't send command to SMTP host
    2023-05-08 06:26:49,673 [main] WARN zingg.common.client.Client - Apologies for this message. Zingg has encountered an error. An error while updating label
    zingg.common.client.ZinggClientException: An error while updating label
    at zingg.common.core.executor.LabelUpdater.processRecordsCli(LabelUpdater.java:97)
    at zingg.common.core.executor.LabelUpdater.execute(LabelUpdater.java:28)
    at zingg.common.client.Client.execute(Client.java:243)
    at zingg.common.client.Client.mainMethod(Client.java:182)
    at zingg.spark.client.SparkClient.main(SparkClient.java:65)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
    Caused by: java.lang.NullPointerException
    at zingg.common.core.executor.LabelUpdater.processRecordsCli(LabelUpdater.java:67)
    ... 16 more
    zingg.common.client.ZinggClientException: An error while updating label
    at zingg.common.core.executor.LabelUpdater.processRecordsCli(LabelUpdater.java:97)
    at zingg.common.core.executor.LabelUpdater.execute(LabelUpdater.java:28)
    at zingg.common.client.Client.execute(Client.java:243)
    at zingg.common.client.Client.mainMethod(Client.java:182)
    at zingg.spark.client.SparkClient.main(SparkClient.java:65)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
    at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
    at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180)
    at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
    at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90)
    at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1039)
    at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1048)
    at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
    Caused by: java.lang.NullPointerException
    at zingg.common.core.executor.LabelUpdater.processRecordsCli(LabelUpdater.java:67)
    ... 16 more

This was found in branch 0.3.5 commit 6bcbb43
fork : https://github.com/zinggAI/zingg-vikas.git
@vikasgupta78 vikasgupta78 added this to the 0.3.5 milestone May 8, 2023
@vikasgupta78
Copy link
Contributor Author

Zingg Model Documentation.pdf

@vikasgupta78
Copy link
Contributor Author

In enum LabelMatchType
for UNDEFINED(ColValues.IS_NOT_KNOWN_PREDICTION, "ARE NOT KNOWN IF MATCH"),
ColValues.IS_NOT_KNOWN_PREDICTION is -1 instead of 2 which is stored in marked data
so what happens is that if in a label updater we try to give a cluster with not sure it gives null pointer exception
as it is not able to get a LabelMatchType instance

@sonalgoyal
Copy link
Member

-1 denotes a pair for which no label has been provided. We also need a 2 for not sure.

We seem to have two classes https://github.com/zinggAI/zingg/blob/0aee8207eb7448525af541ca6c5813c796220d2e/common/core/src/main/java/zingg/common/core/util/LabelMatchType.java and https://github.com/zinggAI/zingg/blob/0aee8207eb7448525af541ca6c5813c796220d2e/common/client/src/main/java/zingg/common/client/util/ColValues.java.

Let us use ColValues consistently

@vikasgupta78 vikasgupta78 mentioned this issue May 8, 2023
Merged
@vikasgupta78
Copy link
Contributor Author

fixed in commit 9aa3d66 pull request #584

@vikasgupta78 vikasgupta78 changed the title label udpater giving exception label updater giving exception May 8, 2023
sonalgoyal added a commit that referenced this issue May 8, 2023
@sonalgoyal
Merge pull request #584 from zinggAI/issue580
a853ccc 
label updater giving exception #580
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
0.3.5
Development

No branches or pull requests
2 participants
@sonalgoyal @vikasgupta78