Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

producer native binary missing for windows from kpl 0.12.0 and above causing crash while trying to create the producer #113

Closed
sshrivastava-incontact opened this issue Jun 1, 2017 · 4 comments
Closed

producer native binary missing for windows from kpl 0.12.0 and above causing crash while trying to create the producer #113

sshrivastava-incontact opened this issue Jun 1, 2017 · 4 comments
Milestone
v0.12.6

Comments

@sshrivastava-incontact
Copy link

sshrivastava-incontact commented Jun 1, 2017 •
edited
Loading

kenesis_producer.exe which is supposed to be packaged in amazon-kinesis-producer-xxxx.jar under amazon-kineses-producer-native-binaries/windows/ is missing from version 0.12.0 and above, attached are the screen shots below.

This missing binary causes the crash at (KinesisProducer.java:807), while trying to create the producer as follows

final KinesisProducer producer = getKinesisProducer();

crash is in (KinesisProducer.java:807) at the following line

_

> byte[] bin = IOUtils.toByteArray(
>                             this.getClass().getClassLoader().getResourceAsStream(root + "/" + os + "/" + executableName));

_

where it is not able to get the resource, following is the log

[

main] INFO com.amazonaws.services.kinesis.producer.KinesisProducer - Extracting binaries to C:\Users\SASHRI1\AppData\Local\Temp\amazon-kinesis-producer-native-binaries
Exception in thread "main" java.lang.RuntimeException: Could not copy native binaries to temp directory C:\Users\SASHRI1\AppData\Local\Temp\amazon-kinesis-producer-native-binaries
at com.amazonaws.services.kinesis.producer.KinesisProducer.extractBinaries(KinesisProducer.java:856)
at com.amazonaws.services.kinesis.producer.KinesisProducer.(KinesisProducer.java:245)
at com.amazonaws.samples.SampleProducer.getKinesisProducer(SampleProducer.java:174)
at com.amazonaws.samples.SampleProducer.main(SampleProducer.java:180)
Caused by: java.lang.NullPointerException
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:462)
at com.amazonaws.services.kinesis.producer.KinesisProducer.extractBinaries(KinesisProducer.java:807)
... 3 more

Maven Dependency :

com.amazonaws amazon-kinesis-producer 0.12.5

KPL 0.10.2 where the binary is present

kpl10 2

KPL 0.12.0 whith missing binary

kpl12 0

KPL 0.12.5 where the binary is not present

kpl0 12 5
@chadp123
Copy link

chadp123 commented Oct 2, 2017

Apparently this documentation is no longer correct?

http://docs.aws.amazon.com/streams/latest/dev/kinesis-kpl-supported-plats.html

The Java package runs without the need to install any additional libraries on the following operating systems:

Linux distributions with kernel 2.6.18 (September 2006) and later
Apple OS X 10.9 and later
Windows Server 2008 and later

Note the KPL is 64-bit only.

@borisbsu
Copy link

From 0.12.0 release notes at https://github.com/awslabs/amazon-kinesis-producer/releases

  • Windows Support is not Available for this Version.

This version of the Kinesis Producer Library doesn't currently support windows. Windows support will be added back at a later date.

@pfifer pfifer added this to the v0.12.6 milestone Nov 13, 2017
pfifer added a commit to pfifer/amazon-kinesis-producer that referenced this issue Nov 14, 2017
@pfifer
Release 0.12.6 of the Amazon Kinesis Producer Library
94ac3c8 
* Added Windows support
  The 0.12.x version now supports Windows.
  The Windows version is currently mastered on the branch `windows`, which will be merged at a later date.
  The build instructions for Windows are currently out of date, and will be updated at a later date.__
  * Issue awslabs#113
  * Issue awslabs#74
  * Issue awslabs#73
* Removed the libc wrapper
  The libc wrapper lowered the required version of glibc.  The KPL is now built with an older version of libc, which removes the need for the wrapper.
  * PR awslabs#139
* Set the minimum required version of macOS to 10.9.
  The KPL is now built against macOS 10.9.
  * Issue awslabs#117
  * PR awslabs#138

* Allow exceptions to bubble to the thread exception handler for Daemon threads.
  Exceptions that occur on daemon threads will now be allowed to propagate to the thread exception handler.  This doesn't provide any additional monitoring or handling of thread death.
  * PR awslabs#112
  * Issue awslabs#111
* Updated `amazon-kinesis-producer-sample` to use the correct properties in its configuration file.
  * PR awslabs#120
  * Issue awslabs#119
* Updated documentation of `AggregationMaxSize` to match actual Kinesis limits.
  * PR awslabs#133
* Added support for setting `ThreadingModel`, and `ThreadPoolSize` using a properties file.
  * PR awslabs#134
  * Issue awslabs#124
* Extracted `IKinesisProducer` from `KinesisProducer` to allow for easier testing.
  * PR awslabs#136
@pfifer pfifer mentioned this issue Nov 14, 2017
Merged
pfifer added a commit that referenced this issue Nov 14, 2017
@pfifer
Release 0.12.6 of the Amazon Kinesis Producer Library (#155)
7f8311b 
* Added Windows support
  The 0.12.x version now supports Windows.
  The Windows version is currently mastered on the branch `windows`, which will be merged at a later date.
  The build instructions for Windows are currently out of date, and will be updated at a later date.__
  * Issue #113
  * Issue #74
  * Issue #73
* Removed the libc wrapper
  The libc wrapper lowered the required version of glibc.  The KPL is now built with an older version of libc, which removes the need for the wrapper.
  * PR #139
* Set the minimum required version of macOS to 10.9.
  The KPL is now built against macOS 10.9.
  * Issue #117
  * PR #138

* Allow exceptions to bubble to the thread exception handler for Daemon threads.
  Exceptions that occur on daemon threads will now be allowed to propagate to the thread exception handler.  This doesn't provide any additional monitoring or handling of thread death.
  * PR #112
  * Issue #111
* Updated `amazon-kinesis-producer-sample` to use the correct properties in its configuration file.
  * PR #120
  * Issue #119
* Updated documentation of `AggregationMaxSize` to match actual Kinesis limits.
  * PR #133
* Added support for setting `ThreadingModel`, and `ThreadPoolSize` using a properties file.
  * PR #134
  * Issue #124
* Extracted `IKinesisProducer` from `KinesisProducer` to allow for easier testing.
  * PR #136
@pfifer
Copy link
Contributor

pfifer commented Nov 16, 2017

Windows support was added as part of the 0.12.6 release.

There is an occasional race condition at startup that may cause failures, but it normally resolves itself relatively quickly. The race condition is on pipe creation, and I'm looking for a better way to manage it.

@pfifer
Copy link
Contributor

pfifer commented May 11, 2018

Closing this, feel free to reopen if you're seeing further issues.

@pfifer pfifer closed this as completed May 11, 2018
@brainstorm brainstorm mentioned this issue Apr 16, 2020
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v0.12.6
Development

No branches or pull requests
4 participants
@borisbsu @pfifer @chadp123 @sshrivastava-incontact