Laravel module to update prices, stock and reviews in affiliate website

Question

I am developing an affiliate website for products similar to www.dtashion.com but I'm facing some issues in updating price, stock and reviews for an individual product in real time. I am using Laravel and MySQL mainly for the development.

My approach:

1. I am currently getting the product details from different websites for the different type of products. Each website supports the different way of providing the product details like in XML format, RESTful API format or simply crawling of their website data.
2. I store each product into one single table with a flag represent the related website like vendor_type = 'FLIPKART' or 'AMAZON' or 'SNAPDEAL' and so on.

My questions are:

1. Is my approach correct?
2. Some websites provide data feed in XML or in RESTful style and for some websites I have to crawl the product details. Now, the problem is at first I can get the product price, stock details and the reviews on their product page or product tag but afterward, how I can update these details on daily basis?
3. What is the best approach in terms of speed, latency, database optimization and etc?

Answer 1

Concerning your 1st and 3rd question: you didn't mention the exact way you do your job, you must mention it in order to judge it, but since its working, then that is a good job, you can always enhance.

Concerning the 2nd question, you can achieve it with task Scheduling in Laravel. If I were you, I will go for the Queuing option.

In the past, you may have generated a Cron entry for each task you needed to schedule on your server. However, this can quickly become a pain, because your task schedule is no longer in source control and you must SSH into your server to add additional Cron entries. Laravel's command scheduler allows you to fluently and expressively define your command schedule within Laravel itself. When using the scheduler, only a single Cron entry is needed on your server.

I hope the answer is beneficial for you, leave a comment if you still have questions.