Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests using Mapbox core styles #5055

Merged
merged 6 commits into from
Aug 3, 2017
Merged

Conversation

brunoabinader
Copy link
Member

@brunoabinader brunoabinader commented Jul 27, 2017

This PR implements support for offline core styles to be used as resource for integration tests in our test suite.

@brunoabinader
Copy link
Member Author

This CI result is a false positive: the bright-v9/z0 expectation I generated from GL JS running on my Linux desktop differs from the one in CI.

@@ -17,6 +17,7 @@ fi
PATH="~/.yarn/bin:$PATH"

yarn
cd test/integration && yarn && cd ../..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The root package.json has this dependency which triggers installation of the integration dependencies automatically. Can you adjust the path munging for local:// URLs so that they load from the root node_modules, making this line unnecessary?

@brunoabinader brunoabinader force-pushed the integration-tests branch 4 times, most recently from 1b94211 to 622a0b3 Compare July 31, 2017 15:07
@brunoabinader
Copy link
Member Author

All tests are passing now, but I'm still catching this message at the end of the test run:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
 2: node::FatalException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
 3: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
 4: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
 5: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
 6: v8::String::WriteUtf8(char*, int, int*, int) const [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
 7: node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
 8: node::Buffer::New(v8::Isolate*, v8::Local<v8::String>, node::encoding) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
 9: node::Buffer::CreateFromString(v8::FunctionCallbackInfo<v8::Value> const&) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
10: v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
11: v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal::Isolate*, v8::internal::(anonymous namespace)::BuiltinArguments<(v8::internal::BuiltinExtraArguments)3>) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
12: v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) [/Users/distiller/mapbox-gl-js/nvm/versions/node/v6.10.1/bin/node]
13: 0x2cdc5092a7```

@brunoabinader
Copy link
Member Author

brunoabinader commented Aug 2, 2017

The JavaScript heap out of memory issue happens only on Linux. The backtrace obtained via gdb is not much helpful (even though I had nodejs-dbg package installed):

(gdb) bt
#0  0x00007ffff6b6f428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff6b7102a in __GI_abort () at abort.c:89
#2  0x00000000010d3f61 in node::Abort() ()
#3  0x00000000010d3f9c in node::OnFatalError(char const*, char const*) ()
#4  0x0000000000955935 in v8::Utils::ReportApiFailure(char const*, char const*) ()
#5  0x0000000000955b3e in v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) ()
#6  0x0000000000c5e1ef in v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) ()
#7  0x0000000000d9d669 in v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) ()
#8  0x00000000009558b0 in v8::internal::String::Flatten(v8::internal::Handle<v8::internal::String>, v8::internal::PretenureFlag) ()
#9  0x000000000097a5fe in v8::String::WriteUtf8(char*, int, int*, int) const ()
#10 0x000000000111bc67 in node::StringBytes::Write(v8::Isolate*, char*, unsigned long, v8::Local<v8::Value>, node::encoding, int*) ()
#11 0x00000000010e9919 in node::Buffer::New(v8::Isolate*, v8::Local<v8::String>, node::encoding) ()
#12 0x00000000010e9b1d in node::Buffer::CreateFromString(v8::FunctionCallbackInfo<v8::Value> const&) ()
#13 0x0000000000981265 in v8::internal::FunctionCallbackArguments::Call(void (*)(v8::FunctionCallbackInfo<v8::Value> const&)) ()
#14 0x00000000009db184 in v8::internal::(anonymous namespace)::HandleApiCallHelper(v8::internal::Isolate*, v8::internal::(anonymous namespace)::BuiltinArguments<(v8::internal::BuiltinExtraArguments)3>) [clone .constprop.187] ()
#15 0x00000000009db86e in v8::internal::Builtin_HandleApiCall(int, v8::internal::Object**, v8::internal::Isolate*) ()
#16 0x0000097d1fe060c7 in ?? ()
#17 0x0000097d1fe06001 in ?? ()
#18 0x00007fffffff6430 in ?? ()
#19 0x0000000300000000 in ?? ()
#20 0x00007fffffff6530 in ?? ()
#21 0x0000097d23bd47be in ?? ()
#22 0x00003f62ec404241 in ?? ()
#23 0x00003f62ec4d40f9 in ?? ()
#24 0x00003f62ec4b8d69 in ?? ()
#25 0x00002f79f59e6cf9 in ?? ()
#26 0x000001373f6aeb39 in ?? ()
#27 0x00007fffffff64c0 in ?? ()
#28 0x0000000000d1687d in v8::internal::ToBooleanIC::ToBoolean(v8::internal::Handle<v8::internal::Object>) ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

From a web search, it turns out this sounds like a common issue:
https://stackoverflow.com/questions/38558989/node-js-heap-out-of-memory

I also tried the heap diff technique from memwatch on the code I've inserted in server.js, but it did not detect any leaks. Thus, my suspicion goes to the fact we're handling a much bigger style data (core styles) than before.

Adding --max-old-space-size=2048 to the node call in test-render fixes this issue. Unfortunately, that parameter is not very well documented (see nodejs/node#7937). However, we're just bumping a few MBytes (from 1400 to 2048). I haven't found a nice way of adding this as a global parameter, but alternatives are welcome :)

@brunoabinader
Copy link
Member Author

@jfirebaugh this should be ready for a 2nd review 👍

Copy link
Contributor

@jfirebaugh jfirebaugh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete test/integration/yarn.lock; otherwise looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants