select * from collection limit 1; is taking a long time to return #364
Comments
|
hi @FGRibreau, it's probably something stupid that we're doing. This cannot be accepted as the normal behavior. Can you share the data set with us or does it have sensitive information ? Either way, optimizing the code is on our todo list. we'll take a look and work on optimizing the code starting with your use case. |
|
Great! I sent an email to @pauldix on that matter (unfortunately I did not had yours). Thanks! |
|
Can you forward the email to support at influxdb dot com or the mailing list |
|
Done ! |
|
Documenting the progress on this story. Currently we don't check the limits at the shard level, which we should do, I opened an issue to track this change #370. Doing a quick hack to test whether this will improve performance, I found out that even with the limit the cpu usage is quite high and the query time didn't improve that much. So we'll have to do some profiling and find the bottle neck. |
|
OK thanks ! Note: below another datapoint from today It now takes 384MB of memory and 16s to retrieve the last point with |
|
We did some profiling and it turns out that the default config is not ideal for your use case. You need to bump the following config option up a little bit, may be to 500 (which I tested locally): [leveldb]
# Maximum mmap open files, this will affect the virtual memory used by
# the process
max-open-files = 500You have 450 columns in your time series, which are stored as different logical time series in leveldb. Reading all columns with the current default limit of 40 max open files, cause leveldb to continuously open, decompress and close the leveldb files. Bumping this number will cause leveldb to cache some information in memory. That said, we still need to propagate limits to the shards so they don't read more points than are actually needed, but making this change to the config will boost performance considerably. |
|
@FGRibreau can you verify the performance after changing this configuration and close the issue if the performance improves and meets your expectations. There's another issue that tracks checking limits at the shard level #370 |
|
@FGRibreau I'm gonne go ahead and close this issue since we're getting ready to release InfluxDB 0.5.3. Please feel free to reopen the issue if you the previous suggested changes to the config file don't work for you. |
|
Hello @jvshahid, Here is what I got with InfluxDB 0.5.3: Then, I stopped influxdb, changed the configuration, restarted it. At idle state influxdb took 7MB of memory, then once the query completed it took 113MB or memory which is quite reasonable. However the response time is better compared to v0.5 (since the number of points was two times higher) [UPDATE] I can't request: Influxdb memory consumption grew to high, I had to kill it. |
|
@jvshahid could you please reopen this issue ? Ths bug is still not fixed |
|
@FGRibreau what version did you use ? You should be using v0.5.4 which has a fix for Issue #341 merged in. |
|
I installed 0.5.5 and that way much better, thanks ! |
We are still testing InfluxDB v0.5 intensively. We are now storing 75973 points.
4 seconds (http request <-> http response) to retrieve the last written point seems way too long. Should we consider this a normal behavior?
I thought that one of InfluxDB goal was to answer queries in less than 1 seconds but I can't find this information on the website anymore... Maybe it was just a dream after all ^^ !
The text was updated successfully, but these errors were encountered: