Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cURL error 35 #287

Closed
vovan47 opened this issue May 11, 2017 · 28 comments
Closed

cURL error 35 #287

vovan47 opened this issue May 11, 2017 · 28 comments
Assignees

Comments

@vovan47
Copy link

vovan47 commented May 11, 2017

Hi guys, I've just got this error:

#0 /library/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php(150): GuzzleHttp\Handler\CurlFactory::createRejection(Object(GuzzleHttp\Handler\EasyHandle), Array) #1 /library/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php(103): GuzzleHttp\Handler\CurlFactory::finishError(Object(GuzzleHttp\Handler\CurlHandler), Object(GuzzleHttp\Handler\EasyHandle), Object(GuzzleHttp\Handler\CurlFactory)) #2 /library/vendor/guzzlehttp/guzzle/src/Handler/CurlHandler.php(43): GuzzleHttp\Handler\CurlFactory::finish(Object(GuzzleHttp\Handler\CurlHandler), Object(GuzzleHttp\Handler\EasyHandle), Object(GuzzleHttp\Handler\CurlFactory)) #3 /library/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php(28): GuzzleHttp\Handler\CurlHandler->__invoke(Object(GuzzleHttp\Psr7\Request), Array) #4 /library/vendor/guzzlehttp/guzzle/src/Handler/Proxy.php(51): GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}(Object(GuzzleHttp\Psr7\Request), Array) #5 /library/vendor/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php(72): GuzzleHttp\Handler\Proxy::GuzzleHttp\Handler\{closure}(Object(GuzzleHttp\Psr7\Request), Array) #6 /library/vendor/guzzlehttp/guzzle/src/Middleware.php(30): GuzzleHttp\PrepareBodyMiddleware->__invoke(Object(GuzzleHttp\Psr7\Request), Array) #7 /library/vendor/guzzlehttp/guzzle/src/RedirectMiddleware.php(68): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Request), Array) #8 /library/vendor/guzzlehttp/guzzle/src/Middleware.php(59): GuzzleHttp\RedirectMiddleware->__invoke(Object(GuzzleHttp\Psr7\Request), Array) #9 /library/vendor/guzzlehttp/guzzle/src/HandlerStack.php(67): GuzzleHttp\Middleware::GuzzleHttp\{closure}(Object(GuzzleHttp\Psr7\Request), Array) #10 /library/vendor/guzzlehttp/guzzle/src/Client.php(275): GuzzleHttp\HandlerStack->__invoke(Object(GuzzleHttp\Psr7\Request), Array) #11 /library/vendor/guzzlehttp/guzzle/src/Client.php(96): GuzzleHttp\Client->transfer(Object(GuzzleHttp\Psr7\Request), Array) #12 /library/vendor/guzzlehttp/guzzle/src/Client.php(104): GuzzleHttp\Client->sendAsync(Object(GuzzleHttp\Psr7\Request), Array) #13 /library/vendor/google/auth/src/HttpHandler/Guzzle6HttpHandler.php(34): GuzzleHttp\Client->send(Object(GuzzleHttp\Psr7\Request), Array) #14 /library/vendor/google/auth/src/OAuth2.php(492): Google\Auth\HttpHandler\Guzzle6HttpHandler->__invoke(Object(GuzzleHttp\Psr7\Request)) #15 /library/vendor/google/auth/src/Credentials/UserRefreshCredentials.php(92): Google\Auth\OAuth2->fetchAuthToken(Object(Google\Auth\HttpHandler\Guzzle6HttpHandler)) #16 /library/vendor/googleads/googleads-php-lib/src/Google/AdsApi/Common/Util/OAuth2TokenRefresher.php(60): Google\Auth\Credentials\UserRefreshCredentials->fetchAuthToken() #17 /library/vendor/googleads/googleads-php-lib/src/Google/AdsApi/AdWords/AdWordsHeaderHandler.php(72): Google\AdsApi\Common\Util\OAuth2TokenRefresher->getOrFetchAccessToken(Object(Google\Auth\Credentials\UserRefreshCredentials)) #18 /library/vendor/googleads/googleads-php-lib/src/Google/AdsApi/Common/AdsSoapClient.php(86): Google\AdsApi\AdWords\AdWordsHeaderHandler->generateHttpHeaders(Object(Google\AdsApi\AdWords\AdWordsSession)) #19 /library/vendor/googleads/googleads-php-lib/src/Google/AdsApi/AdWords/v201702/mcm/ManagedCustomerService.php(103): Google\AdsApi\Common\AdsSoapClient->__soapCall('get', Array) #20 /home/ubuntu/cleoo/application/Modules/marketingengine/src/Service/NewAdwords/Helpers/AccountHelper.php(62): Google\AdsApi\AdWords\v201702\mcm\ManagedCustomerService->get(Object(Google\AdsApi\AdWords\v201702\cm\Selector)) ...

Any ideas how it could be fixed?

@fiboknacky
Copy link
Member

Hello @vovan47

It looks like the error message itself is cropped and only stack trace is shown.
Could you please re-produce the issue and fetch the error message as well?

Best,
Knack

@vovan47
Copy link
Author

vovan47 commented May 16, 2017 via email

@fiboknacky
Copy link
Member

fiboknacky commented May 16, 2017

Hi Vladimir,

My assumption is you're hitting the limit of your systems (as hinted in this post).
If you're hitting the rate limit of the AdWords API, there will be errors returned from the servers.

Can you explain more if you use multithreading too?
And does this happen regularly or intermittently?

Knack

@vovan47
Copy link
Author

vovan47 commented May 19, 2017 via email

@vovan47
Copy link
Author

vovan47 commented May 20, 2017 via email

@fiboknacky
Copy link
Member

Hi Vladimir,

You don't need to create a session every time you change the services to use.
It's true that you need to create a new session if you would like to use a different clientCustomerId.
But for the AdWords services, it's a different thing.

For example, in this line, AdWordsServices is created and used in the example.
You can reuse this object to create a new service.

Again, the only scenario that you have to create a new session object is when you change clientCustomerId.

Best,
Knack

@vovan47
Copy link
Author

vovan47 commented May 22, 2017 via email

@fiboknacky
Copy link
Member

Hello Vladimir,

That will still generate some unclosed fopen's. But implementing session
caching by customer ID should improve situation with fopen.
But still, we noticed that config file is not released/closed correctly,
because fopen number persisted even when my script stopped running. Maybe
there's something in library code you can improve?

There is no complication in the logic here, as the config file is read by the native PHP function: parse_ini_file() in this line.
It's supposed to close the file properly and return the result as an array, so I don't think we can do anything much on this.
(We don't even hold any file pointers of config files in session objects).

And so on, repeat X times (500, 1000 etc), every time manager account could
be different (though totally we have 3-5 of them).

To clarify, when it repeats for 500 times, you create 3-5 sessions each time?
It looks like they should be cached as you mention for the sessions of the same clientCustomerId.
I'm not sure how you implement this part but garbage collection should work if your sessions are not used anymore.

Based on what I've researched, it looks like sometimes the garbage collector is not aggressive enough and you can still run into the problem that memory isn't freed up properly.
In that case, you could try:

  • gc_collect_cycles()
  • Do the unset() on variables you don't use anymore. This would help the garbage collector to free up memory.

I've tried using the above two functions in the past and it did help in some cases.

Best,
Knack

@vovan47
Copy link
Author

vovan47 commented May 22, 2017 via email

@fiboknacky
Copy link
Member

Hi Vladimir,

I'll try to reproduce on my side as well so I'd like to ask for more information about your scenarios.

Could you please confirm the following?

  • What's your PHP version, OS, default_socket_timeout? If you have any special configurations of PHP, could you please share too?
  • In your scenario, could you please share a bit more about the following?
    • You says your repeat the process for X times, e.g., X = 500, 1000. What's the maximum of this X?
    • Provided that totally you have not so many sessions objects in the system, in each iteration, the factor that you're changing is only services, e.g., from using AdGroupService to CampaignService? Are there any factors you're varying here?
    • What do you do in each iteration? Services get()? mutate()? or both?
  • You mentioned that it looks like some files are not closed after your script ends. How long does it take to close? Or are they never closed and you need to kill them manually each time?

Thanks in advance.
Knack

@vovan47
Copy link
Author

vovan47 commented May 24, 2017 via email

@fiboknacky
Copy link
Member

Hi Vladimir,

How does your APP_BASE_PATH look like?
Could you please debug to see what's the type and value of APP_BASE_PATH . '/dynamic/logs/soap.log', when passed into AdsLoggerFactory?

It looks like false is passed instead of a string so this exception is thrown in this line.

Best,
Knack

@vovan47
Copy link
Author

vovan47 commented May 25, 2017 via email

@vovan47
Copy link
Author

vovan47 commented May 25, 2017 via email

@fiboknacky
Copy link
Member

Hi Vladimir,

Thanks for more insights!

One question I have for your explanation:

Well, as I have to create session for each customer ID, there will be
number of sessions equal to count of customer accounts, so 4'000+.

But they all don't persist at the same time, right?
From what you explain, in each iteration, for a certain clientCustomerId (or equivalently session), you get all campaigns, ad groups, ads and keywords and save them into the database.
This mean the session object should be out of scope after your loop moves to the next iteration?

Totally, there shouldn't be 4,000 session objects existing at the same time?
(To be precise, it can be more than just a few session objects though, depending on how the garbage collector works)

And about your usage pattern, it'd be much better to use structure reports.
This is also listed up in our best practices.
The SOAP services aren't designed to call intensively to sync with your backends. Reporting works much better in this case.

Cheers,
Knack

@fiboknacky
Copy link
Member

Hi Vladimir,

As for your earlier question, looking more closely, I see that the issue lies in the initialization of batch job logger.
It seems you've not provided any configuration file path for the batch job logger, thus the default stream is used, which is stderr (at this line).

But somehow, stderr isn't working at that time, so fopen() returns false and fails the StreamHandler() constructor.
How does your configuration file (or string) look like?

Best,
Knack

@vovan47
Copy link
Author

vovan47 commented May 25, 2017 via email

@vovan47
Copy link
Author

vovan47 commented May 25, 2017 via email

@fiboknacky fiboknacky self-assigned this May 25, 2017
@fiboknacky
Copy link
Member

Hello Vladimir,

Does this happen all the time (you can't get through at all) or intermittently after many API requests?

Thanks.
Knack

@vovan47
Copy link
Author

vovan47 commented May 25, 2017 via email

@vovan47
Copy link
Author

vovan47 commented May 26, 2017 via email

@vovan47
Copy link
Author

vovan47 commented May 26, 2017 via email

@fiboknacky
Copy link
Member

Hi Vladimir,

Thanks for helping investigate this issue.
One thing that's not clear to me is the ConnectException.

Where did you get the below error?

Exception 'GuzzleHttp\Exception\ConnectException' with message 'cURL error 35: error:02001018:system library:fopen:Too many open files (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)

The library's SOAP services don't make use of the Guzzle, so I wonder how the above error is generated.
But the suggestion from the Guzzle issue makes sense.
I'll check in the library to see where we can apply the suggestion.

Best,
Knack

@vovan47
Copy link
Author

vovan47 commented May 26, 2017 via email

@fiboknacky
Copy link
Member

Hi Vladimir,

I see. That happens during the token refreshing process, which is a part of Google Auth library, not during the SOAP calls.
The WSDL error is another separate issue (which might be involved in some ways).

Let me think how we can resolve this.

By the way, is it possible to have your curl version as well?

Thanks.
Knack

@vovan47
Copy link
Author

vovan47 commented May 29, 2017 via email

dklimkin added a commit that referenced this issue May 30, 2017
PHP SoapClient creates keep-alive connections but never reuse them.
Disabling this behavior. Should resolve issues #298 and #287.
@vovan47
Copy link
Author

vovan47 commented Jun 1, 2017 via email

@fiboknacky
Copy link
Member

Hi Vladimir,

I've just made a release for v27.1.0, which also includes the latest version of the API (v201705).

I'll close this issue for now. Feel free to report if you find other different issues.
Enjoy coding!

Cheers,
Knack

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants