/    Sign up×
Community /Pin to ProfileBookmark

fetch and insert ALOT of data into DB

Could you guys please help me structure my code?

I have this system coded in PHP already but its a TOTALLY mess becasue i dont know how to structure it in a good way. This time I am going to code the script in Javascript with axios and this time I want to do it right.

I fetch alot of data from alot of different APIs and API endpoints. Then I insert all this data into my DB.

the way i built it in PHP was _something_ like this:

**File1 – a cron job**
calling all the functions to grab the data. Every single function gets called 5 times with different dates. Then the data is inserted into my db

“`
funcs = [‘func1′,’func2’,func3’….] // 150 or so
dates = [‘2020-12-20’, ‘2020-12-21’,’2020-12-22’….] // 7 atm
data = []

for(i<funcs.length; i++) {
for(a<dates.length; a++) {
// grabbing one functions JSON data
data.push(a[i](dates[a]))
}
}

/*
connect to db
insert query1
insert query2
query3
query4
….
query150
*/

// close db con
“`

**file2**
A class that includes all the functions. aka func1() func2() func3()… which each returns a JSON object

the cron job is called once a day

to post a comment
JavaScript

1 Comments(s)

Copy linkTweet thisAlerts:
@alexweefsDec 24.2020 — Everyone here at [Dean's Tank, Inc.](http://www.deanstank.net/), from the service professionals to the office staff, is dedicated to exceeding your expectations. Serving the Twin Cities and all of Minnesota, Dean’s Tank is the most experienced residential petroleum tank contractor in the Midwest. With over 20 years of experience, we will work with your individual needs and will complete the project according to your specifications.
×

Success!

Help @cvem spread the word by sharing this article on Twitter...

Tweet This
Sign in
Forgot password?
Sign in with TwitchSign in with GithubCreate Account
about: ({
version: 0.1.9 BETA 4.25,
whats_new: community page,
up_next: more Davinci•003 tasks,
coming_soon: events calendar,
social: @webDeveloperHQ
});

legal: ({
terms: of use,
privacy: policy
});
changelog: (
version: 0.1.9,
notes: added community page

version: 0.1.8,
notes: added Davinci•003

version: 0.1.7,
notes: upvote answers to bounties

version: 0.1.6,
notes: article editor refresh
)...
recent_tips: (
tipper: @Yussuf4331,
tipped: article
amount: 1000 SATS,

tipper: @darkwebsites540,
tipped: article
amount: 10 SATS,

tipper: @Samric24,
tipped: article
amount: 1000 SATS,
)...