Failing to send messages - unable to parse JSON #1

numericOverflow · 2018-04-19T13:52:01Z

Recently, xvoice++ stopped sending messages, and it appears as the attempt to send the SMS is not getting back a JSON object around line 156 in sendGVMessage() . I was able to pull my android logs and I've attached a screenshot of majority of the error messages.

I think I've seen this in the pyGoogleVoice (unofficial python GVoice API) and I'm wondering if google is sending the plain text error message instructing the user to go to their site and prove their not a robot

Is it possible to get a try/catch block to output the error message to debug? Right now it just dies with an unhandled exception so no visibility to what's actually coming back.

I'm not a android developer, but I'd be more than happy to help you test/debug or provide any logs you need to potentially diagnose. I've got a rooted phone so have some leeway on accessing debug logs, etc.

BehindTheMath · 2018-04-19T14:44:32Z

It's been a long time since I looked at this project, but I'll see what I can do.

It might be faster to use a proxy like Fiddler or Charles to intercept the traffic on a computer and inspect it.

numericOverflow · 2018-04-19T16:13:11Z

Ah, good idea. I pulled this snippet of the conversation between xvoice++ and google:

<------
GET /voice/request/user HTTP/1.1
Host: www.google.com
User-Agent: Dalvik/2.1.0 (Linux; U; Android)
Accept-Encoding: gzip, deflate
Connection: keep-alive
Accept: /
Authorization: GoogleLogin auth=[[REDACTED]].

------>
HTTP/1.1 302 Moved Temporarily
Location: https://voice.google.com
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Date: Thu, 19 Apr 2018 15:07:51 GMT
Expires: Thu, 19 Apr 2018 15:07:51 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alt-Svc: hq=":443"; ma=2592000; quic=51303433; quic=51303432; quic=51303431; quic=51303339; quic=51303335,quic=":443"; ma=2592000; v="43,42,41,39,35"
Transfer-Encoding: chunked

<HTML>
<HEAD>
<TITLE>Moved Temporarily</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Moved Temporarily</H1>
The document has moved <A HREF="https://voice.google.com">here</A>.
</BODY>
</HTML>

<------
POST /voice/sms/send/ HTTP/1.1
Host: www.google.com
User-Agent: Dalvik/2.1.0 (Linux; U; Android)
Accept-Encoding: gzip, deflate
Connection: keep-alive
Accept: /
Authorization: GoogleLogin auth=[[REDACTED]].
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: 52

phoneNumber=%2B15555555555&sendErrorSms=0&text=TextMessageHere

------>
HTTP/1.1 405 HTTP method POST is not supported by this URL
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Date: Thu, 19 Apr 2018 15:07:54 GMT
Expires: Thu, 19 Apr 2018 15:07:54 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alt-Svc: hq=":443"; ma=2592000; quic=51303433; quic=51303432; quic=51303431; quic=51303339; quic=51303335,quic=":443"; ma=2592000; v="43,42,41,39,35"
Transfer-Encoding: chunked

<HTML>
<HEAD>
<TITLE>HTTP method POST is not supported by this URL</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>HTTP method POST is not supported by this URL</H1>
<H2>Error 405</H2>
</BODY>
</HTML>

BehindTheMath · 2018-04-22T03:09:14Z

Based on my testing, it looks like the GV API is checking the user-agent string, and sending back an error message if it detects an Android user-agent.

I'm going to try patch it by sending a blank user-agent. In my testing that appears to work.

Of course, if this change is because Google is trying to block uses like this, we may have to switch to spoofing a desktop user-agent. It's also possible that they can change the API completely.

BehindTheMath · 2018-04-22T03:15:51Z

Is it possible to get a try/catch block to output the error message to debug? Right now it just dies with an unhandled exception so no visibility to what's actually coming back.

I don't think this is a good idea in general, since the response could potentially contain sensitive information, like authorization codes.

You can always clone the repo, add your own debugging code, and recompile the APK.

BehindTheMath · 2018-04-22T03:47:03Z

I'm going to try patch it by sending a blank user-agent. In my testing that appears to work.

Apparently a setting a blank user-agent with Ion doesn't actually set it. Instead, I set it to spoof Chrome 65 on Windows 10 x64.

I uploaded a new build here. Please try it and tell me if it works.

numericOverflow · 2018-04-22T04:41:34Z

I just installed your dev version and sent a couple test messages, which seemed to send ok. I'll give it a better workout tomorrow and let you know how it goes.

BehindTheMath · 2018-04-24T02:52:12Z

@numericOverflow Can I assume it's working as expected?

numericOverflow · 2018-04-24T12:44:53Z

@BehindTheMath - It's sort of working. I don't see that original JSON error in the logs any more, but xvoice+ isn't reliably retrieving my messages - some come through, others don't, but all show in the official gvoice app. I'm trying to track down exactly what's going on to reproduce consistently.

About the only thing I can find it that I seem to get this error in my logs:

Conscrypt initialization failed.
java.lang.NoClassDefFoundError: Failed resolution of: Lcom/google/android/gms/security/ProviderInstaller;
	at com.koushikdutta.ion.conscrypt.ConscryptMiddleware.initialize(ConscryptMiddleware.java:58)
	at com.koushikdutta.ion.conscrypt.ConscryptMiddleware.initialize(ConscryptMiddleware.java:75)
	at com.koushikdutta.ion.conscrypt.ConscryptMiddleware.getSocket(ConscryptMiddleware.java:103)
	at com.koushikdutta.async.http.AsyncHttpClient.executeAffinity(AsyncHttpClient.java:301)
	at com.koushikdutta.async.http.AsyncHttpClient.access$100(AsyncHttpClient.java:51)
	at com.koushikdutta.async.http.AsyncHttpClient$1.run(AsyncHttpClient.java:190)
	at com.koushikdutta.async.AsyncServer.lockAndRunQueue(AsyncServer.java:740)
	at com.koushikdutta.async.AsyncServer.runLoop(AsyncServer.java:758)
	at com.koushikdutta.async.AsyncServer.run(AsyncServer.java:658)
	at com.koushikdutta.async.AsyncServer.access$800(AsyncServer.java:44)
	at com.koushikdutta.async.AsyncServer$14.run(AsyncServer.java:600)
Caused by: java.lang.ClassNotFoundException: Didn't find class "com.google.android.gms.security.ProviderInstaller" on path: DexPathList[[zip file "/data/app/io.behindthemath.xvoiceplus-1/base.apk"],nativeLibraryDirectories=[/vendor/lib64, /system/lib64]]
	at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
	... 11 more
	Suppressed: java.lang.ClassNotFoundException: Didn't find class "com.google.android.gms.security.ProviderInstaller" on path: DexPathList[[dex file "/data/dalvik-cache/xposed_XResourcesSuperClass.dex", dex file "/data/dalvik-cache/xposed_XTypedArraySuperClass.dex"],nativeLibraryDirectories=[/vendor/lib64, /system/lib64]]
		at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
		at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
		... 12 more
		Suppressed: java.lang.ClassNotFoundException: com.google.android.gms.security.ProviderInstaller
			at java.lang.Class.classForName(Native Method)
			at java.lang.BootClassLoader.findClass(ClassLoader.java:781)
			at java.lang.BootClassLoader.loadClass(ClassLoader.java:841)
			at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
			... 13 more
		Caused by: java.lang.NoClassDefFoundError: Class not found using the boot class loader; no stack available

BehindTheMath · 2018-04-24T18:13:40Z

@numericOverflow Do you have Google Play Services or GApps installed?

Edit: I see you commented here about this error. Based on @koush's comment here and the comment here, it seems like this error just means Ion isn't using Play Services for SSL, and is falling back to use the system's implementation, so there should be no difference in functionality.

I suppose I could add some try...catch blocks to suppress the error from showing.

numericOverflow · 2018-04-25T01:41:31Z

I do have Google play services, I'm running an LG G4 from VZW that's rooted, but otherwise mostly stock Verizon Lillipop.

I think you're right, I just can't find any other errors in the logs to explain why xvoice+ isn't working consistently, that was all I could find.

BehindTheMath · 2018-04-25T12:59:10Z

It's been a while since I personally used XVoice++, but if I remember correctly, I had the same issue and I wasn't able to pin down the cause. The only thing I can suggest is to enable all the sync options.

numericOverflow · 2018-04-26T02:15:37Z

Hmm... I've already got all the sync options enabled. I guess I'll keep trying to track down something helpful to diagnose the issue.

tb-303 · 2018-04-29T15:18:56Z

Ya, I have sync issues with the dev build, too :( Still on api 23, no issues prior to this changeup for years

BehindTheMath · 2018-04-29T16:33:39Z

@tb-303 You're having issues with incoming or outgoing or syncing? It sounds like everyone else is just having issues with incoming.

tb-303 · 2018-05-03T15:05:17Z

No, just issues with incoming sync.

nathanmerrill · 2018-05-13T06:12:44Z

I tried the new .apk, and I'm also having issues receiving. To be explicit:

If I send from an SMS app, the recipient gets it. The google voice app doesn't show that it was ever sent.
However, if they reply, the google voice app does show the reply, but the SMS app doesn't show any reply.

I'd love to help with this, but I have no idea how to get a local environment setup to test this. The only process I can think of is to install Xposed on the android emulator, and then this app, which seems like an ordeal.

nathanmerrill · 2018-05-15T04:48:46Z

Ok, so I may have made some progress (or I could be on the wrong track entirely).

I think that the method that XVoicePlus hooks into doesn't exist anymore (com.google.android.apps.voice.backends.gcm::a). I exported the Hangouts app, decompiled it, and as far as I can tell, there's no class by that name. I poked around in the Jar, and I think that the new method to listen to is com.google.android.gms.gcm.GcmReceiver::a, but I'm not confident on that.

I wish I could be more confident in the above, but I'm unable to get debugging or logging working at handleLoadPackage(), which apparently happens super early. If anybody has any suggestions on how to get that working, that would be awesome.

nathanmerrill · 2018-05-15T14:31:48Z

More progress: I believe that the above methods are being called, but I don't think that logging works at that time.

The fundamental problem is that the request in GoogleVoiceManager::retrieveMessages(). It is also getting a JSON error, which is preventing us from reading messages.

There's also a bug relating to Sync poll timer, making it not work (hookBroadcastPermissionCheck would change the UID of the USER_POLL action (it was just supposed to change the SMS_DELIVER action)). I've fixed that locally, but since syncing doesn't work, I haven't submitted a PR.

nathanmerrill · 2018-05-16T01:36:58Z

I fixed the user agent on all of the queries, but it appears that the query "https://www.google.com/voice/request/messages" doesn't return text messages anymore. It returns voice messages, but there's no more attribute for texts. I tried capturing traffic, but I'm not sure how to bypass SSL (I've tried Packet Capture, but it simply says "No data")

BehindTheMath · 2018-05-16T17:37:09Z

@nathanmerrill Thank you for working on debugging this. I'm busy with a few other projects, so currently I don't have much time for this one.

I think that the method that XVoicePlus hooks into doesn't exist anymore (com.google.android.apps.voice.backends.gcm::a). I exported the Hangouts app, decompiled it, and as far as I can tell, there's no class by that name. I poked around in the Jar, and I think that the new method to listen to is com.google.android.gms.gcm.GcmReceiver::a, but I'm not confident on that.

Do you mean Hangouts(com.google.android.talk) or Google Voice(com.google.android.apps.googlevoice)? XVoice never hooked Hangouts; it only use Google Voice. I checked GV v5.9.191668881 last week, and it still appears to have the com.google.android.apps.voice.backends.gcm.GcmListenerService class that XVoicePlus hooks. As you mentioned, there is also com.google.android.gms.gcm.GcmReceiver, but I'm not sure what that class does, and at which point in the message processing sequence it is used.

I wish I could be more confident in the above, but I'm unable to get debugging or logging working at handleLoadPackage(), which apparently happens super early. If anybody has any suggestions on how to get that working, that would be awesome.

handleLoadPackage() is called by Xposed early in the boot sequence, as it initializes the module. As far as I know, there's no way to debug it. Logging should work, however, and as you can see from the current code, there are several logging calls which used to work.

More progress: I believe that the above methods are being called, but I don't think that logging works at that time.

I'm not sure about this, since as I mentioned, logging should work.

The fundamental problem is that the request in GoogleVoiceManager::retrieveMessages(). It is also getting a JSON error, which is preventing us from reading messages.

I believe this is due to the user agent string, as mentioned upthread. This should be fixed in the test build.

There's also a bug relating to Sync poll timer, making it not work (hookBroadcastPermissionCheck would change the UID of the USER_POLL action (it was just supposed to change the SMS_DELIVER action)). I've fixed that locally, but since syncing doesn't work, I haven't submitted a PR.

That's interesting. I guess there should be a check for which action it is. Can you post a snippet of what you changed?

I fixed the user agent on all of the queries, but it appears that the query "https://www.google.com/voice/request/messages" doesn't return text messages anymore. It returns voice messages, but there's no more attribute for texts.

I would have to double-check, but I thought it worked when I tried it last week.

I tried capturing traffic, but I'm not sure how to bypass SSL (I've tried Packet Capture, but it simply says "No data")

You need to use a custom root certificate from a network capture program like Fiddler or Charles so you can decrypt the HTTPS traffic.

nathanmerrill · 2018-05-16T17:49:13Z

Do you mean Hangouts or Google Voice?

You are totally right: I was looking at hangouts.

I would have to double-check, but I thought it worked when I tried it last week.

If you hit that URL (in your browser, assuming you are logged in), it should auto download the JSON.

I was unable to get Fiddler or Charles to work. I may have done something wrong, but it appears that hangouts is detecting it somehow.

On a side note: I've never been able to use this, as I just recently found it. I have the hangouts app installed, and assumed that that was the Google Voice app. That's clearly not the case :)

BehindTheMath · 2018-05-16T18:04:03Z

Can you try with the GV app and see what still doesn't work?

nathanmerrill · 2018-05-16T18:11:20Z

Yes, I definitely can tonight. I don't have my machine with Android Studio setup in front of me :)

nathanmerrill · 2018-05-16T20:53:20Z

@BehindTheMath I reinstalled the APK above, and incoming SMS still isn't working. It's not an error, either, we simply aren't getting the notification. (There's no log for "Handling intent for action INCOMING_VOICE")

BehindTheMath · 2018-05-16T20:59:20Z

@BehindTheMath I reinstalled the APK above, and incoming SMS still isn't working. It's not an error, either, we simply aren't getting the notification. (There's no log for "Handling intent for action INCOMING_VOICE")

Right, that's the issue everyone else is having.

It appears to me that either com.google.android.apps.voice.backends.gcm.GcmListenerService.a() isn't being called, or the hook isn't being called, but I don't know why.

Does time-based syncing work for you?

nathanmerrill · 2018-05-16T21:06:24Z

It works on my patched version (but my patch broke outgoing SMS).

…

On Wed, May 16, 2018, 2:59 PM BehindTheMath ***@***.***> wrote: @BehindTheMath <https://github.com/BehindTheMath> I reinstalled the APK above, and incoming SMS still isn't working. It's not an error, either, we simply aren't getting the notification. (There's no log for "Handling intent for action INCOMING_VOICE") Right, that's the issue everyone else is having. It appears to me that either com.google.android.apps.voice.backends.gcm.GcmListenerService.a() isn't being called, or the hook isn't being called, but I don't know why. Does time-based syncing work for you? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC0XP3d6I4-taQJISTfeKxiSFyw6U7d7ks5tzJMogaJpZM4Tb1fc> .

BehindTheMath · 2018-05-16T21:07:20Z

It works on my patched version (but my patch broke outgoing SMS).

What did you change?

nathanmerrill · 2018-05-16T21:14:29Z

In XVoicePlus::hookBroadcastPermissionCheck I checked the third parameter (Intent) and checked what the intent name was. I can get you the code in a hour and a half.

…

On Wed, May 16, 2018, 3:07 PM BehindTheMath ***@***.***> wrote: It works on my patched version (but my patch broke outgoing SMS). What did you change? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC0XP1svJZZ9gIHxLNcrRsOdE8dPu2oYks5tzJUKgaJpZM4Tb1fc> .

nathanmerrill · 2018-05-16T23:10:14Z

`final int ARGUMENT_INDEX_CALLING_UID = 14;
final int ARGUMENT_INDEX_INTENT = 2;
int callingUid = (int) param.args[ARGUMENT_INDEX_CALLING_UID];
Intent intent = (Intent) param.args[ARGUMENT_INDEX_INTENT];

// Get our UID
int appUid = AndroidAppHelper.currentApplication().getPackageManager()
.getApplicationInfo(XVOICE_PLUS_PACKAGE, PackageManager.GET_META_DATA).uid;
// If the broadcast is from us
if ("android.provider.Telephony.SMS_DELIVER".equals(intent.getAction()) && (boolean) callStaticMethod(UserHandle.class, "isSameApp", callingUid, appUid)) {

Log.d(TAG, "Hooking broadcast permissions: Overriding callingUid "
		+ callingUid + " with Process.PHONE_UID (UID 1001)");

// Spoof the broadcast as if it's coming from PHONE_UID, so the system will let it through
param.args[ARGUMENT_INDEX_CALLING_UID] = Process.PHONE_UID;

}
`

This is the code so far. I can make an APK if you want.

Anyways, I think the error that requires addressing is: BroadcastQueue is showing the following error: Permission Denial: receiving Intent { act=android.provider.Telephony.SMS_DELIVER flg=0x10 (has extras ) } to com.google.android.talk/com.google.android.apps.hangouts.sms.SmsDeliverReceiver requires android.permission.RECEIVE_SMS due to sender io.behind themath.xvoiceplus (uid 1001)

nathanmerrill · 2018-05-16T23:28:40Z

Ok, that error was actually pretty trivial to fix. Now dealing with Error accessing registered_accounts from GV SharedPreferences

nathanmerrill · 2018-05-17T04:27:43Z

Ok, so apparently, the path to the GV preferences changed. The fix is in GCMListenerServiceHook line 104 to final String registeredAccounts = gvSharedPrefs.getString("accounts", null); (change registered_accounts to accounts).

Great news though, there are still more issues. https://www.google.com/voice/sms/send/ is now returning a 500 error for me.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing to send messages - unable to parse JSON #1

Failing to send messages - unable to parse JSON #1

numericOverflow commented Apr 19, 2018

BehindTheMath commented Apr 19, 2018

numericOverflow commented Apr 19, 2018

BehindTheMath commented Apr 22, 2018

BehindTheMath commented Apr 22, 2018

BehindTheMath commented Apr 22, 2018

numericOverflow commented Apr 22, 2018

BehindTheMath commented Apr 24, 2018

numericOverflow commented Apr 24, 2018

BehindTheMath commented Apr 24, 2018 •

edited

Loading

numericOverflow commented Apr 25, 2018

BehindTheMath commented Apr 25, 2018

numericOverflow commented Apr 26, 2018

tb-303 commented Apr 29, 2018 •

edited

Loading

BehindTheMath commented Apr 29, 2018

tb-303 commented May 3, 2018

nathanmerrill commented May 13, 2018

nathanmerrill commented May 15, 2018

nathanmerrill commented May 15, 2018

nathanmerrill commented May 16, 2018

BehindTheMath commented May 16, 2018

nathanmerrill commented May 16, 2018

BehindTheMath commented May 16, 2018

nathanmerrill commented May 16, 2018

nathanmerrill commented May 16, 2018

BehindTheMath commented May 16, 2018

nathanmerrill commented May 16, 2018 via email

BehindTheMath commented May 16, 2018

nathanmerrill commented May 16, 2018 via email

nathanmerrill commented May 16, 2018

nathanmerrill commented May 16, 2018

nathanmerrill commented May 17, 2018

Failing to send messages - unable to parse JSON #1

Failing to send messages - unable to parse JSON #1

Comments

numericOverflow commented Apr 19, 2018

BehindTheMath commented Apr 19, 2018

numericOverflow commented Apr 19, 2018

BehindTheMath commented Apr 22, 2018

BehindTheMath commented Apr 22, 2018

BehindTheMath commented Apr 22, 2018

numericOverflow commented Apr 22, 2018

BehindTheMath commented Apr 24, 2018

numericOverflow commented Apr 24, 2018

BehindTheMath commented Apr 24, 2018 • edited Loading

numericOverflow commented Apr 25, 2018

BehindTheMath commented Apr 25, 2018

numericOverflow commented Apr 26, 2018

tb-303 commented Apr 29, 2018 • edited Loading

BehindTheMath commented Apr 29, 2018

tb-303 commented May 3, 2018

nathanmerrill commented May 13, 2018

nathanmerrill commented May 15, 2018

nathanmerrill commented May 15, 2018

nathanmerrill commented May 16, 2018

BehindTheMath commented May 16, 2018

nathanmerrill commented May 16, 2018

BehindTheMath commented May 16, 2018

nathanmerrill commented May 16, 2018

nathanmerrill commented May 16, 2018

BehindTheMath commented May 16, 2018

nathanmerrill commented May 16, 2018 via email

BehindTheMath commented May 16, 2018

nathanmerrill commented May 16, 2018 via email

nathanmerrill commented May 16, 2018

nathanmerrill commented May 16, 2018

nathanmerrill commented May 17, 2018

BehindTheMath commented Apr 24, 2018 •

edited

Loading

tb-303 commented Apr 29, 2018 •

edited

Loading