Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Found a Google Translate endpoint that doesn't require an API key. #268

Closed
SuperSonicHub1 opened this issue Jan 3, 2021 · 62 comments
Closed
Assignees
Labels

Comments

@SuperSonicHub1
Copy link

This isn't a bug report, just a sharing of some findings while looking through minified source code.

While digging through the source code of Google's Google Dictionary Chrome extension, which has support for translating via Google Translate, I found the endpoint they use in order to do just that. Since googletrans frequently runs into 5xx errors, it might be useful to switch off to another endpoint, although this one is also annoyingly touchy with 403s.

Breakdown

Endpoint: https://clients5.google.com/translate_a/t

Query Parameters

Query Parameter Default Notes
client dict-chrome-ex Needs to be dict-chrome-ex or else you'll get a 403 error.
sl auto Designates the source language of the text to translate.
tl (none) Designates the destination language of the text to translate.
q (none) Text to translate

Example Response

URL: https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q=bonjour

{
  "sentences": [
    {
      "trans": "Hello",
      "orig": "bonjour",
      "backend": 1
    }
  ],
  "dict": [
    {
      "pos": "interjection",
      "terms": [
        "Hello!",
        "Hi!",
        "Good morning!",
        "Good afternoon!",
        "How do you do?",
        "Hallo!",
        "Hullo!",
        "Welcome!"
      ],
      "entry": [
        {
          "word": "Hello!",
          "reverse_translation": [
            "Bonjour!",
            "Salut!",
            "Tiens!",
            "Allô!"
          ],
          "score": 0.7316156
        },
        {
          "word": "Hi!",
          "reverse_translation": [
            "Salut!",
            "Bonjour!",
            "Hé!"
          ],
          "score": 0.084690653
        },
        {
          "word": "Good morning!",
          "reverse_translation": [
            "Bonjour!"
          ],
          "score": 0.065957151
        },
        {
          "word": "Good afternoon!",
          "reverse_translation": [
            "Bonjour!"
          ],
          "score": 0.02749503
        },
        {
          "word": "How do you do?",
          "reverse_translation": [
            "Bonjour!",
            "Salut!",
            "Ça marche?"
          ]
        },
        {
          "word": "Hallo!",
          "reverse_translation": [
            "Bonjour!",
            "Tiens!",
            "Salut!",
            "Allô!"
          ]
        },
        {
          "word": "Hullo!",
          "reverse_translation": [
            "Tiens!",
            "Allô!",
            "Salut!",
            "Bonjour!"
          ]
        },
        {
          "word": "Welcome!",
          "reverse_translation": [
            "Salut!",
            "Bonjour!",
            "Soyez le bienvenu!"
          ]
        }
      ],
      "base_form": "Bonjour!",
      "pos_enum": 9
    }
  ],
  "src": "fr",
  "alternative_translations": [
    {
      "src_phrase": "bonjour",
      "alternative": [
        {
          "word_postproc": "Hello",
          "score": 1000,
          "has_preceding_space": true,
          "attach_to_next_token": false
        }
      ],
      "srcunicodeoffsets": [
        {
          "begin": 0,
          "end": 7
        }
      ],
      "raw_src_segment": "bonjour",
      "start_pos": 0,
      "end_pos": 0
    }
  ],
  "confidence": 0.96875,
  "ld_result": {
    "srclangs": [
      "fr"
    ],
    "srclangs_confidences": [
      0.96875
    ],
    "extended_srclangs": [
      "fr"
    ]
  },
  "query_inflections": [
    {
      "written_form": "bonjour",
      "features": {
        "gender": 1,
        "number": 2
      }
    },
    {
      "written_form": "bonjours",
      "features": {
        "gender": 1,
        "number": 1
      }
    },
    {
      "written_form": "bonjour",
      "features": {
        "number": 2
      }
    },
    {
      "written_form": "bonjours",
      "features": {
        "number": 1
      }
    }
  ]
}
@ssut
Copy link
Owner

ssut commented Jan 4, 2021

This seems like a great discovery! I'll take a look at this shortly. Thank you so much!

@ssut ssut pinned this issue Jan 4, 2021
@SuperSonicHub1
Copy link
Author

@ssut No problem!

@SuperSonicHub1
Copy link
Author

@theonefoster Glad to be of use!

@shatteringlass
Copy link

Stopped working for me (403)

@SuperSonicHub1
Copy link
Author

SuperSonicHub1 commented Jan 13, 2021

@shatteringlass Mentioned that in the initial post:

it might be useful to switch off to another endpoint, although this one is also annoyingly touchy with 403s.

That's why I want this endpoint to be seamlessly integrated into googletrans, with it switching between endpoints if one is facing 4xx/5xx errors.

@Animenosekai
Copy link

I found another endpoint within the source code of one of the google translate extensions on VSCode too.

"https://translate.googleapis.com/translate_a/single?client=gtx&dt=t + params"
// where the params are:
{
  "sl": source language,
  "tl": destination language,
  "q": the text to translate
}

The results looks something like this:

[[["こんにちは、今日はお元気ですか?","Hello, how are you today?",null,null,3,null,null,[[]
]
,[[["9588ca5d94759e1e85ee26c1b641b1e3","kgmt_en_ja_2020q3.md"]
]
]
]
]
,null,"en",null,null,null,null,[]
]

for the query: https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=ja&q=Hello, how are you today?

And something like this:

[[["Bonjour","Hello",null,null,1]
]
,null,"en",null,null,null,null,[]
]

for the query: https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=fr&q=Hello

When using it only to translate things, I would use it like so:

from json import loads
from requests import get
request_result = get("https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=fr&q=Hello")
translated_text = loads(request_result.text)[0][0][0]
print(translated_text)

@SuperSonicHub1
Copy link
Author

@Animenosekai Awesome, another endpoint to add. We should probably create a way to house all of these endpoints through one API, perhaps an interface that all of the endpoints implement?

@NawtJ0sh
Copy link

NawtJ0sh commented Jan 26, 2021

What am I doing wrong?

word = 'لماذا تفعل هذا'
try:
    request_result = requests.post("https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q="+ word).json()
    
    print(request_result)
    print('[In English]: '+ request_result['alternative_translations'][0]['alternative'][0]['word_postproc'])
    print('[Language Dectected]: ' + request_result['src'])
except:
     traceback.print_exc()

Output:

[In English]: Ù „Ù… Ø§Ø ° ا ت٠عل Ù ‡ Ø ° ا
[Language Dectected]: co

why is it outputting it like that?

@SuperSonicHub1
Copy link
Author

@NawtJ0sh Why are you making a POST request? It's a GET request you should be making. How does that even return a response?

@NawtJ0sh
Copy link

NawtJ0sh commented Jan 26, 2021

No idea, it does the exact same thing with requests.get()

@Animenosekai
Copy link

Animenosekai commented Jan 26, 2021

No idea, it does the exact same thing with requests.get()

If you try to open it in a browser it seems to work fine:
https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q=%D9%84%D9%85%D8%A7%D8%B0%D8%A7%20%D8%AA%D9%81%D8%B9%D9%84%20%D9%87%D8%B0%D8%A7

{"sentences":[{"trans":"Why are you doing this","orig":"لماذا تفعل هذا","backend":1},{"src_translit":"limadha tafeal hdha"}],"src":"ar","alternative_translations":[{"src_phrase":"لماذا تفعل هذا","alternative":[{"word_postproc":"Why are you doing this","score":1000,"has_preceding_space":true,"attach_to_next_token":false}],"srcunicodeoffsets":[{"begin":0,"end":14}],"raw_src_segment":"لماذا تفعل هذا","start_pos":0,"end_pos":0}],"confidence":1,"ld_result":{"srclangs":["ar"],"srclangs_confidences":[1],"extended_srclangs":["ar"]}}

Might be a problem with the decoding, try to use the json module yourself (print request.text() first and print the json formatted version json.loads(request.text()) to see the differences)

It might be because your terminal doesn't support arabic or something like that

Also maybe try to url encode the text before sending it (and use GET as it should not work with POST)

@d4n3436
Copy link

d4n3436 commented Jan 26, 2021

After some testing with the request headers, I found the solution for the garbled text.
Simply set the User-Agent header to the one that Google Chrome uses.

Example:

import requests
word = 'لماذا تفعل هذا'
url = "https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl=en&q=" + word
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36'
}

try:
    request_result = requests.get(url, headers=headers).json()
    
    print(request_result)
    print('[In English]: ' + request_result['alternative_translations'][0]['alternative'][0]['word_postproc'])
    print('[Language Dectected]: ' + request_result['src'])
except:
     pass

Response:

{'sentences': [{'trans': 'Why are you doing this', 'orig': 'لماذا تفعل هذا', 'backend': 1}, {'src_translit': 'limadha tafeal hdha'}], 'src': 'ar', 'alternative_translations': [{'src_phrase': 'لماذا تفعل هذا', 'alternative': [{'word_postproc': 'Why are you doing this', 'score': 1000, 'has_preceding_space': True, 'attach_to_next_token': False}], 'srcunicodeoffsets': [{'begin': 0, 'end': 14}], 'raw_src_segment': 'لماذا تفعل هذا', 'start_pos': 0, 'end_pos': 0}], 'confidence': 1, 'ld_result': {'srclangs': ['ar'], 'srclangs_confidences': [1], 'extended_srclangs': ['ar']}}
[In English]: Why are you doing this
[Language Dectected]: ar

@SuperSonicHub1
Copy link
Author

@d4n3436 How interesting. It's almost like a form of copy protection.

@Animenosekai
Copy link

@d4n3436 How interesting. It's almost like a form of copy protection.

Maybe that they are retrieving the User-Agent to process some analytics in the bg and that without it it breaks something.

But, I mean, still weird that it gives a gibberish response and that it accepted the POST request

@d4n3436
Copy link

d4n3436 commented Jan 27, 2021

Maybe that they are retrieving the User-Agent to process some analytics in the bg and that without it it breaks something.

It's weird...

If you don't use User-Agent, the response will have an incorrect encoding (ASCII).
If you use a banned User-Agent like curl/7.37.1, you'll get a 403 error page.
But if you use a web browser User-Agent, the response will have a correct encoding (UTF-8).

About the POST request, it seems that the API endpoint allows using POST requests using query parameters instead of a body. This also works with PUT, PATCH, DELETE and OPTIONS (basically all common methods).

@NawtJ0sh
Copy link

if anyone wants to try and get this Bing translator to work that'd be awesome

import requests

url = 'https://www.bing.com/ttranslatev3'

post_header = {}
post_header['Host'] = 'www.bing.com'
post_header['User-Agent'] = 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0'
post_header['Accept'] = '*/*'
post_header['Accept-Language'] = 'en-US,en;q=0.5'
post_header['Accept-Encoding'] = 'gzip, deflate'
post_header['Referer'] = 'https://www.bing.com/'
post_header['Content-Type'] = 'application/x-www-form-urlencoded'
post_header['Connection'] = 'keep-alive'

parameters_payload = {'IG' : '839D27F8277F4AA3B0EDB83C255D0D70', 'IID' : 'translator.5033.3'}
data_payload = {'text':'Platypus', 'from':'en', 'to':'es'}
resp = requests.post(url, headers=post_header, params=parameters_payload, data=data_payload)

print(resp.json())

@Animenosekai
Copy link

@NawtJ0sh Microsoft might have changed their API since I get back:

{'statusCode': 400} # if this was a http status code, means bad request --> might be because the params don't work anymore

(which is weird since the HTTP status code is 200)

@d4n3436
Copy link

d4n3436 commented Jan 31, 2021

@NawtJ0sh I made a simple API wrapper for Bing Translator in C# some months ago. You can use it and port it.
BingTranslatorApi.cs

@NawtJ0sh
Copy link

@NawtJ0sh I made a simple API wrapper for Bing Translator in C# some months ago. You can use it and port it.
BingTranslatorApi.cs

Nice! if you could try it with the Yandex translator, that'd be cool!

@NawtJ0sh
Copy link

@NawtJ0sh Microsoft might have changed their API since I get back:

{'statusCode': 400} # if this was a http status code, means bad request --> might be because the params don't work anymore

(which is weird since the HTTP status code is 200)

All that was changed was the 'from' to 'fromLang' haha

@d4n3436
Copy link

d4n3436 commented Jan 31, 2021

Nice! if you could try it with the Yandex translator, that'd be cool!

I actually made one, but unfortunately Yandex has a captcha protection that is triggered after a few uses of their API.

@Animenosekai
Copy link

@NawtJ0sh Microsoft might have changed their API since I get back:

{'statusCode': 400} # if this was a http status code, means bad request --> might be because the params don't work anymore

(which is weird since the HTTP status code is 200)

All that was changed was the 'from' to 'fromLang' haha

Lmao, yea I just checked and it worked!

[{'detectedLanguage': {'language': 'en', 'score': 1.0}, 'translations': [{'text': 'Ornitorrinco', 'to': 'es', 'sentLen': {'srcSentLen': [8], 'transSentLen': [12]}}]}]

@Animenosekai
Copy link

if anyone wants to try and get this Bing translator to work that'd be awesome

Well here you go:

from json import loads
from requests import post
HEADERS = {
    "Host": "www.bing.com",
    "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0",
    "Accept": "*/*",
    "Accept-Language": "en-US,en;q=0.5",
    "Accept-Encoding": "gzip, deflate",
    "Referer": "https://www.bing.com/",
    "Content-Type": "application/x-www-form-urlencoded",
    "Connection": "keep-alive"
}
PARAMS = {'IG' : '839D27F8277F4AA3B0EDB83C255D0D70', 'IID' : 'translator.5033.3'}

def translate(text, to, fromLang="auto-detect"):
    """
    Translates the given text to the given language using Microsoft's Bing Translation API (ttranslatev3)
    """
    request = post("https://www.bing.com/ttranslatev3", headers=HEADERS, params=PARAMS, data={'text': str(text), 'fromLang': str(fromLang), 'to': str(to)})
    if request.status_code < 400:
        try:
            return loads(request.text)[0]["translations"][0]["text"]
        except:
            return None
    else:
        return None

def language(text):
    """
    Gives you back the ISO 639-1 Alpha-2 language code the text has been written in using Microsoft's Bing Translation API (ttranslatev3)

    > The output is a tuple with the language code and the score (confidence)
    """
    request = post("https://www.bing.com/ttranslatev3", headers=HEADERS, params=PARAMS, data={'text': str(text), 'fromLang': "auto-detect", 'to': "en"})
    if request.status_code < 400:
        try:
            detectedLanguage = loads(request.text)[0]["detectedLanguage"]
            return detectedLanguage["language"], detectedLanguage["score"]
        except:
            return None
    else:
        return None

Just call

translate("<your text>", "<the ISO 639-1 Alpha-2 language codes (I guess) of the output language>")

Or if you want to know the language of a given text:

language("<your text>")

And it gives you back the result or None if an error occurred.

Example:

>>> translate("Hello", "ja")
'こんにちは'
>>> translate("Hello", "fr")
'Bonjour'
>>> translate("Hello", "fr", "en")
'Bonjour'
>>> language("Hello")
('en', 1.0)
>>> language("Hola")
('es', 1.0)

@NawtJ0sh
Copy link

NawtJ0sh commented Feb 1, 2021

Here is the yandex translate, I worked on last night its just like the bing.com one i posted.

import requests

url = 'https://translate.yandex.net/api/v1/tr.json/translate?id=1308a84a.6016deed.0c4881a2.74722d74657874-3-0&srv=tr-text&lang=en&reason=auto&format=text'

post_header = {}
post_header['Accept'] = '*/*'
post_header['Accept-Encoding'] = 'gzip, deflate'
post_header['Accept-Language'] = 'en-US,en;q=0.9'
post_header['Cache-Control'] = 'no-cache'
post_header['Connection'] = 'keep-alive'
post_header['Content-Type'] = 'application/x-www-form-urlencoded'
post_header['Host'] = 'translate.yandex.com'
post_header['Referer'] = 'https://translate.yandex.com/'
post_header['User-Agent'] = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36'


data_payload = {'text': search_str, 'options': '4'}

resp = requests.get(url, headers=post_header, data=data_payload).json()

print(str(resp) + '\n')

@Animenosekai
Copy link

Animenosekai commented Feb 1, 2021

@NawtJ0sh

I just made a repository which groups all of the APIs if you want (called translate)

https://github.com/Animenosekai/translate

@NawtJ0sh
Copy link

Nice you fixed it!! Thank you.

@x011
Copy link

x011 commented Sep 23, 2021

import requests

 """
l = language
s = sentence
"""

def trans1(l, s):
	return requests.get(f"https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=auto&tl={l}&q={s}").json()[0][0][0]

def trans2(l, s):
	return requests.get(f"https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=auto&tl={l}&q={s}").json()["sentences"][0]["trans"]

@nquenault
Copy link

Error from server : Too many request :'(

@SuperSonicHub1
Copy link
Author

Closing this issue as our research was never implemented in this repo. I think it's best to assume that py-googletrans is pretty much done. Still need a translation library? Check out these:

Please stop using this issue as a support thread; tired of getting everyone's emails. Maintainers, I'd recommend you lock this thread.

@evrial
Copy link

evrial commented Feb 22, 2022

Why you confusing people with broken package? Update the readme and unpublish from the pip. Thats it

@vloverar
Copy link

It seems that now the two "free endpoints" of Google Translate API don't respond in a nice way anymore. For single words translations, they used to respond with a detailed list of possible translations. Now they only give you back a single one (the most common one) and I can't find a way to change this.

Have a look for "body" translated from EN to IT ("corpo", "organismo", "organo", "carrozzeria", ...)

Free endpoint 1: https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=en&tl=it&q=body

Free endpoint 2: https://translate.googleapis.com/translate_a/single?client=gtx&dt=t&sl=en&tl=it&q=body

Any ideas?

@CoolCat467
Copy link

I don't believe it would be possible to get detailed lists without using a different endpoint, because in the applications these endpoints are in, they don't need all the possible translations.

@lokinmodar
Copy link

lokinmodar commented Apr 16, 2022

I've found this alternative endpoint for google:
Free endpoint: https://translate.google.com/m?hl=en&sl=en&tl=pt&ie=UTF-8&prev=_m&q=Text
The caveat? the response is an HTML page. It is easily parseable though and the translation is much better than the ones they are spitting through the simple json endpoints above

@Animenosekai
Copy link

@lokinmodar The only caveat is that it's the mobile version of Google Translate which tends to yield worse translations

@Animenosekai
Copy link

Animenosekai commented Apr 16, 2022

@lokinmodar The only caveat is that it's the mobile version of Google Translate which tends to yield worse translations (based on previous observations, this might have changed and this page is slightly different than the normal Google Translate mobile page)

@lokinmodar
Copy link

@lokinmodar The only caveat is that it's the mobile version of Google Translate which tends to yield worse translations (based on previous observations, this might have changed and this page is slightly different than the normal Google Translate mobile page)

i noticed this is much more faithful to the translations I used to get with the https://clients5.google.com/translate_a/t?client=dict-chrome-ex&sl=en&tl=it&q=body endpoint before they screwed it up

@d4n3436
Copy link

d4n3436 commented Apr 16, 2022

I made a comparison of all (known) Google Translate API endpoints based on my findings while making my translation library, this may be useful if you don't know which API endpoint to use.

  • https://clients5.google.com/translate_a no longer seems to return faithful translations; only returns the translated text.
  • https://translate.googleapis.com/translate_a/single still returns faithful translations and some info about the translation (source language, translation confidence and transliteration if available).
  • https://translate.google.com/_/TranslateWebserverUi/data/batchexecute only returns a faithful translation with the X-Goog-BatchExecute-Bgr, the algorithm that creates this header doesn't seem to be deciphered yet. This is the endpoint that returns the most info (source language, translation confidence, transliteration, alternative translations, definitions, examples, etc.).
  • https://translate.google.com/m returns a faithful translation, but no extra info. Returns HTML instead of JSON so parsing is required.

In conclusion, the second endpoint is currently the best free Google Translate API endpoint, unless someone manages to decipher the X-Goog-BatchExecute-Bgr header.

@NawtJ0sh
Copy link

NawtJ0sh commented Jun 4, 2022

https://clients5.google.com/translate_a/single?dj=1&dt=t&dt=sp&dt=ld&dt=bd&client=dict-chrome-ex&sl=auto&tl=en&q=kanker

this is what the url is like now (on the Google Dictionary Extension)

@steam3d
Copy link

steam3d commented Jul 12, 2022

Does anyone understand what is "score". Looks like it show how popular translation. I can't understand how to convert it to dashes like google translate does.
image

@ffreemt
Copy link

ffreemt commented Oct 20, 2022

https://translate.google.com/_/TranslateWebserverUi/data/batchexecu seems no longer to work, 404 now.

@ffreemt
Copy link

ffreemt commented Oct 20, 2022

https://translate.google.com/_/TranslateWebserverUi/data/batchexecute still OK. https://translate.google.cn/_/TranslateWebserverUi/data/batchexecute no longer works because translate.google.cn is dead (redirected to translate.google.com.hk in fact).

@lipichang
Copy link

Hello,everyone
someone who can find translate from image.

@tkefauver
Copy link

I know this is an older thread but just wanted to warn people that https://translate.googleapis.com/translate_a/single has issues with punctuation and will just halt translations (like if it encounters an '!').

https://clients5.google.com/translate_a/t works great though! Thank you @SuperSonicHub1 !

@RemyJouni
Copy link

RemyJouni commented Sep 24, 2024

I know this is an older thread but just wanted to warn people that https://translate.googleapis.com/translate_a/single has issues with punctuation and will just halt translations (like if it encounters an '!').

https://clients5.google.com/translate_a/t works great though! Thank you @SuperSonicHub1 !

I have found that https://translate.googleapis.com/translate_a/single works perfectly, even when it encounters punctuation such as "!".

The only drawback is that it splits the translation into multiple parts if the query is too long, which requires us to join them back together.

It also provides far better translations compared to https://clients5.google.com/translate_a/t.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests