Shopify Integration [FEATURE]

[cgsmith]

Is your feature request related to a problem? Please describe.
Currently I have a PHP scripts folder that just runs cron jobs and pulls down open orders via Shopify's API. This is problematic because it doesn't pull down updates.

Describe the solution you'd like
This feature relates to #129 and #98 - I don't think it depends on it.

As a user:

• I want to be able to add a Shopify integration through the Shipwise website
• I want to use OAuth to setup the integration
• I want to be able to "pause" the integration
• I might have multiple Shopify stores
• I may want to specify a specific status to pull orders in on (Paid/Unfulfilled as an example).

Technically:

• The integration should use webhooks to receive the order change
• We should keep a raw payload in a table as show in Order workflow discussion [FEATURE] #129
• If the order cannot be parsed for some reason we should send an email to the customer (possibly a summary email, this can be a separate task)
• We will be adding other oauth flows
• Should be able to use the ecommerce adapters like this
  • Feel free to refactor as this code isn't used in production (Duda adapter is)
• I started using Integration to handle this. You can refactor or change - this is not used in production

Note:

I was planning on using hookdeck for all integrations but we can reevaluate this later.

References:

• https://shopify.dev/docs/apps/webhooks
• https://shopify.dev/docs/apps/auth/oauth

[bohdan-vorona]

@cgsmith

Currently I have a PHP scripts folder that just runs cron jobs and pulls down open orders via Shopify's API.

Is it possible to get them? Just in case.

[cgsmith]

Yes ill invite you to the scripts.

[cgsmith]

Curious if we should use a service like this? https://www.codat.io/

I'm okay if we do not. External services can be expensive and unreliable. I invited you the scripts.

[cgsmith]

This is the bit of code that pulls orders down: https://github.com/CGSmith-LLC/shipwise-backend/blob/master/cron/countrypet.php#L139-L146

[bohdan-vorona]

@cgsmith thanks

[cgsmith]

Here is a Codat link to checkout... maybe we can leverage this to integrate with many platforms at once?

Just not sure on what it would cost: https://docs.codat.io/docs/get-started-api

Most customers have Shopify accounts.

[cgsmith]

@bohdan-vorona - forget I mentioned Codat. Let's just integrate directly that way we can control our flow.

If we need to use a service like Codat in the future we will. Sorry for the confusion!

[bohdan-vorona]

@cgsmith ok, no problem 👌🏻

[bohdan-vorona]

@cgsmith

As a user:

I want to be able to add a Shopify integration through the Shipwise website
I want to use OAuth to setup the integration
I want to be able to "pause" the integration
I might have multiple Shopify stores
I may want to specify a specific status to pull orders in on (Paid/Unfulfilled as an example).

And

We will be adding other oauth flows

0. As I see we have a separate Yii app shopify with its own config. Why do we need this? Also, we have some tables and models (shopify_app, shopify_webhook) but in my case they are empty. Why do we need this?

1. Shall I continue using the existing page /integration and existing code+tables? I mean, as the basis.

2. Am I allowed to edit the current logic? For instance, you'd like to add the "Pause" state. Also, I'd probably upgrade something here )

3. Am I right that I need to add the ability to connect/disconnect a service using OAuth on the /integrarion page? For the moment, you'd like to have the ability to connect/disconnect Shopify on the page /integration?

4. If leave the page /integration available only for the admin, then using OAuth he would be able to authorize only himself. I'm not sure it's correct. Can you please clarify this moment? I have an assumption that we should have a new Integrations page available for all users where they will be able to connect their Shopify using OAuth. Or shall I keep using the form below (attached image) where we connect a customer with an integration?

5. If we leave the current page /integration and upgrade it (add Shopify, add the ability to pause an integration, add the ability to use OAuth, etc.), then here, in the form, we keep linking customers with integrations, correct?

[cgsmith]

@bohdan-vorona

As I see we have a separate Yii app shopify with its own config. Why do we need this? Also, we have some tables and models (shopify_app, shopify_webhook) but in my case they are empty. Why do we need this?

This can be deleted and removed. It was going to be used for a Shopify App but was never fully tested or finished.

Shall I continue using the existing page /integration and existing code+tables? I mean, as the basis.
Am I allowed to edit the current logic? For instance, you'd like to add the "Pause" state. Also, I'd probably upgrade something here )

Yes. It is not used in production feel free to change all.

Am I right that I need to add the ability to connect/disconnect a service using OAuth on the /integrarion page? For the moment, you'd like to have the ability to connect/disconnect Shopify on the page /integration?

Yes

If leave the page /integration available only for the admin, then using OAuth he would be able to authorize only himself. I'm not sure it's correct. Can you please clarify this moment? I have an assumption that we should have a new Integrations page available for all users where they will be able to connect their Shopify using OAuth. Or shall I keep using the form below (attached image) where we connect a customer with an integration?

The /integration page will be available for all customers/users after testing.

If we leave the current page /integration and upgrade it (add Shopify, add the ability to pause an integration, add the ability to use OAuth, etc.), then here, in the form, we keep linking customers with integrations, correct?

Feel free to redesign the flow or add any forms. This will mainly be customer facing for them to connect ecommerce sites.

[bohdan-vorona]

@cgsmith Thanks 👌

[cgsmith]

@bohdan-vorona feel free to create other issues if you feel like we need to separate some of the code out.

[bohdan-vorona]

@cgsmith Since /integration isn't used, maybe I'd separate a task with Integrations (like a basic functionality with tables, models, forms, etc.) itself and then Shopify integration as a separate task extended from Integrations (basic logic).

Will do it tomorrow)

[bohdan-vorona]

@cgsmith

1. Update master.
2. Switch to https://github.com/CGSmith-LLC/shipwise-api/tree/feature/shopify-integration. The branch is a child branch of https://github.com/CGSmith-LLC/shipwise-api/tree/feature/e-commerce-integrations.
3. Update composer.
4. Apply the new migrations.
5. Visit intro-docs\ecommerce-platfroms.md and read the guide.

Pages:

1. /ecommerce-platform for Platforms management (must be Admin).
2. /ecommerce-integration/index for users to manage their integrations with Shopify.

[bohdan-vorona]

@cgsmith

I have some questions to clarify:

1/ Shopify API doesn't provide full information about the Shipping address of test orders. I have no idea why (maybe during the production it will be ok). But our Address model has some required fields. What shall I do?

Have a look: https://prnt.sc/Rh3GHfR1GGVw <-> https://prnt.sc/VeLuRvqD-37b

A) For the moment, I set "Not provided" for the required fields if they are not provided by Shopify API - https://prnt.sc/xiTk3t_kAnR5 We can keep this logic.

B) Skip such Orders (don't parse them) - I mean, Orders without needed fields in the Shipping address.

2/ Also, there are lots of test Items without SKU but this field is required on our side. What shall I do?

A) For the moment, I set "Not provided" for the required fields if they are not provided by Shopify API - https://prnt.sc/-8DI6PdUVMpw We can keep this logic.

B) Skip such Items (don't parse them). But, in this case, there can be easily a situation when we have removed all Items and the Order has no Items at all. Save the Order with no Items or don't save (skip)?

3/ Here https://github.com/CGSmith-LLC/shipwise-backend/blob/master/src/CGSMITH/Shopify.php you set Carrier as FedEx. It seems that Shopify API doesn't provide such information. Shall I do the same (set default carrier) or set it as NULL?

4/ At the moment I pull only orders with the fulfilment status = unfulfilled. But you wrote: I may want to specify a specific status to pull orders in on (Paid/Unfulfilled as an example) So, in Shop Settings, there must be an ability to set the statuses? The default value is ALL?

5/ The user can go to his Shop and uninstall our Shopify App. In this case, we will not be able to pull orders. What shall I do in this case?

A) Just remove the shop on our side.

B) Send an email.

C) Add a new status, something like Uninstalled on our side with the ability to re-connect the shop again.

D) Your option)

6/ We have users and customers. During connecting a Shop, shall I display here all our customers or only customers related to the current user ID via the table user_customer?

---

[cgsmith]

1. I wonder if this is due to how the call is made or authorized for PII. I will get you access to our test Shopify site. 1a. This is fine for now but might need to be adjusted in the future. 1b. I was thinking we could send orders that don't parse to the users email who setup the integration. Maybe a summary email for orders that didn't parse? Any thoughts? 2. Ignore and put in an error report? Thoughts? I'm okay with just ignoring as well. 3. Setting as NULL should be fine. Shipwise will contain this logic eventually. 4. This should be changed on the integration setup. We will receive all web hooks but only act if it matches the status. 5. Options b and c 6. Only customers relating to the user.

…

On Thu, Mar 2, 2023, 1:08 PM Bohdan ***@***.***> wrote: @cgsmith <https://github.com/cgsmith> *I have some questions to clarify:* *1/* Shopify API doesn't provide full information about the Shipping address of test orders. I have no idea why (maybe during the production it will be ok). But our Address model has some required fields. What shall I do? Have a look: https://prnt.sc/Rh3GHfR1GGVw <-> https://prnt.sc/VeLuRvqD-37b *A)* For the moment, I set "Not provided" for the required fields if they are not provided by Shopify API - https://prnt.sc/xiTk3t_kAnR5 We can keep this logic. *B)* Skip such Orders (don't parse them) - I mean, Orders without needed fields in the Shipping address. *2/* Also, there are lots of test Items without SKU but this field is required on our side. What shall I do? *A)* For the moment, I set "Not provided" for the required fields if they are not provided by Shopify API - https://prnt.sc/-8DI6PdUVMpw We can keep this logic. *B)* Skip such Items (don't parse them). But, in this case, there can be easily a situation when we have removed all Items and the Order has no Items at all. Save the Order with no Items or don't save (skip)? *3/* Here https://github.com/CGSmith-LLC/shipwise-backend/blob/master/src/CGSMITH/Shopify.php you set Carrier as FedEx. It seems that Shopify API doesn't provide such information. Shall I do the same (set default carrier) or set it as NULL? *4/* At the moment I pull only orders with the fulfilment status = unfulfilled. But you wrote: *I may want to specify a specific status to pull orders in on (Paid/Unfulfilled as an example)* So, in Shop Settings, there must be an ability to set the statuses? The default value is ALL? *5/* The user can go to his Shop and *uninstall our Shopify App*. In this case, we will not be able to pull orders. What shall I do in this case? *A)* Just remove the shop on our side. *B)* Send an email. *C)* Add a new status, something like Uninstalled on our side with the ability to re-connect the shop again. *D)* Your option) *6/* We have users and customers. During connecting a Shop, shall I display here all our customers or only customers related to the current user ID via the table user_customer? [image: customer-drop-down] <https://user-images.githubusercontent.com/5950639/222527210-1fbf3cc1-57a3-40a2-91c2-a4f245a65180.png> ------------------------------ [image: shopify-order-example] <https://user-images.githubusercontent.com/5950639/222527542-0a966635-52cf-4a28-b3a4-e4a57ab37695.png> — Reply to this email directly, view it on GitHub <#137 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAELFIRJX4LXZQ7R22B6DNLW2DV4JANCNFSM6AAAAAAU5APK4E> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bohdan-vorona]

@cgsmith Thanks for your reply!

1. I will try to read more about this in their API docs.
   1b. Yes, we can do it. Anyway, I propose to store not parsed orders in our system somewhere.

2. Ok, in addition to my previous comment about storing, I will save them. And we will be able to send error reports or do any other actions with such orders.

3-6. Ok.

[cgsmith]

@bohdan-vorona - I sent the invite to your email for partners.shopify.com

I gave access to the chris-test-me-now site... which you can use however you want. Just a dev site.

Blue Nest Beef you should have access to as well for a production site. It is a friend's site so you can install the app or test on that for webhooks if you'd like. Let me know if you need me to test or research anything for you.

[bohdan-vorona]

@cgsmith Ok, thanks 👌

[bohdan-vorona]

@cgsmith Status update:

1. I found the reason why the Shipping Address was empty - Shopify Apps require additional requests for such data.
2. Implemented the logic of switching to the status Uninstalled when the user has uninstalled our App.
3. I send a notification once a shop is uninstalled.
4. Implemented the logic of reconnecting uninstalled shops.
5. Customers drop-down only for the current user.
6. Implemented the logic when the user (while connecting a shop) can choose: Order Statuses, Financial Statuses, and Fulfillment Statuses.
7. I added logging for orders we will pull from platforms like Shopify. In fact, the table will store the status of parsing (success or failure), the reason why not parsed, raw order, and some other needed data. In fact, at the moment it's just a DB table but can be used for example for reports with not parsed orders and preparing or so.

---

I was not able to test on Blue Nest Beef since only an approved app can be installed in a real shop. So, I'm continuing testing on my test shops.

[cgsmith]

I will have time to test this week as well.

[bohdan-vorona]

@cgsmith Status update:

I'm finishing webhooks processing. I mean, processing of these webhooks received from Shopify:

        'orders/create',
        'orders/cancelled',
        'orders/updated',
        'orders/delete',
        'orders/fulfilled',
        'orders/partially_fulfilled',
        'orders/paid',
        'app/uninstalled'

Mandatory:

        'customers/data_request',
        'customers/redact',
        'shop/redact',

---

Some moments:

• orders/create -> I create an order in our system. Correct?
• orders/cancelled -> I set the status=Cancelled in our system. Correct?
• orders/updated -> I update the fields we have in our system. Correct?
• orders/delete -> I delete the order in our system. Correct?
• app/uninstalled -> I set the shop integration's status to Uninstalled and send a notification.

What shall I do with these webhooks (see the list below)? Change the status or...? If change the status, then what accordance? Or just mark them as success and do nothing?

        'orders/fulfilled',
        'orders/partially_fulfilled',
        'orders/paid',

---

I'm testing the logic on my test Shopify App and my two test Shopify Shops. I wanted to use your App's credentials but you didn't provide access for me. If you would like me to test using your App's credentials as well, please grant me corresponding access )

Finishing the webhooks processing and it would be nice if you check and test it locally since there is a good chunk of logic ) I will let you know. Meanwhile, I will switch to Order attributes.