/    Sign up×
Community /Pin to ProfileBookmark

Web-API many queries

I’m working on a new project and struggling with the right setup.
Topic is a database of products of a specific branch/industry sector, where producers can register their products (Article No., Description, Price, etc.) on a website (csv-upload for example).
Different software providers of the branch/industry sector should then be able to implement the API so the users can lookup the products.
Because of the potential huge amount of queries and the different programs I’m not really sure which type of API should be used for the best performance and compatibility of different software developed in different languages.
My first thought would be a REST API which can be handled by nearly all development environments, would you recommend that?
And which database type would be the best for data amount of more than 2-3 M. entries (additional with different table joins because of separation of manufacturer and items). And many different queries of the software users also the same time.
**1) Which type of API is recommended? (REST-API?)
2) Which type of database is recommended for performance?
3) Is there a server sided must have?**

Thank you very much in advance for your thoughts,
Christian

to post a comment

7 Comments(s)

Copy linkTweet thisAlerts:
@NogDogDec 03.2021 — > @Change#1640138 additional with different table joins because of separation of manufacturer and items

I would re-think that, instead including a manufacturer table with foreign keys to it where applicable.

> @Change#1640138 And which database type would be the best for data amount of more than 2-3 M. entries

Any of the major players can handle that amount fine (as long as you index things correctly). MySQL would be fine, though I prefer PostgreSQL for reasons other than performance (maybe mostly because I work with it every day?), though it performs at least equally well.

> @Change#1640138 Which type of API is recommended? (REST-API?)

REST could make sense if simply because so many server-side language frameworks support it out of the box. If, however, you expect requests to be fairly complex, I might look into JSON:API as an option. (I'm not terribly familiar with it, but my understanding is it's particularly useful for dealing with complex requests.)
Copy linkTweet thisAlerts:
@ChangeauthorDec 04.2021 — Thank you very much for your answer.

> @NogDog#1640145 I would re-think that, instead including a manufacturer table with foreign keys to it where applicable.
>
I thought about two tables because of the sql normalization.

So for example table 1
``<i>
</i>[article_no, description, price, manufacturer]
[123, Screw galvanized 3.5x35mm, 12.48, 2587]<i>
</i>
`</CODE>

table 2
<CODE>
`<i>
</i>[manufacturer_no, name]
[2587, ScrewWorld Inc.]<i>
</i>
``


> @NogDog#1640145 Any of the major players can handle that amount fine (as long as you index things correctly). MySQL would be fine, though I prefer PostgreSQL for reasons other than performance (maybe mostly because I work with it every day?), though it performs at least equally well.

I checked Postgres before, problem with postgress is the memory lack,

Every connection forks a new process with 10MB memory allocation. So for a huge amount of queries MySQL seems to be better, but i dont know much about other databases.

For what i read until now ORacle SQL would be better than MySQL but needs a paid license. So i will take MySQL i think.

> @NogDog#1640145 REST could make sense if simply because so many server-side language frameworks support it out of the box. If, however, you expect requests to be fairly complex, I might look into JSON:API as an option. (I'm not terribly familiar with it, but my understanding is it's particularly useful for dealing with complex requests.)

I will have a look on JSON:API, thanks for that hint :)
Copy linkTweet thisAlerts:
@NogDogDec 04.2021 — > @Change#1640170 problem with postgress is the memory lack

Hmm...not familiar with this. The app I work on uses PostgreSQL (on an AWS RDS instance) hit by 3 web servers with up to 1500 requests per minute in peak hours (and more than a few of those requests make more than one query). Again, not saying Postgres is better than MySQL, but that as far as I know you should be fine with either (unless one has some specific capability you want to leverage).

PS: We do run Postgres with the PGBouncer connection handler, which does help prevent occasional lags when new connections would need to be spun up otherwise.
Copy linkTweet thisAlerts:
@ChangeauthorDec 04.2021 — I see, maybe you prevent the speed lack through PGBouncer, im not that familir with that.

Im not in programming of software, is every software (like C++, C#, vb.net, VB, Swift etc.) able to work with a JSON REST-API? And how they call it? With a simple http request?

Copy linkTweet thisAlerts:
@NogDogDec 04.2021 — @Change#1640182

My first google hit for "c# json api" led me to https://github.com/json-api-dotnet/JsonApiDotNetCore/

It looks like in their case they layer json:api on top of REST, so I guess they're not mutually exclusive. 🤷
Copy linkTweet thisAlerts:
@ChangeauthorDec 04.2021 — Most REST-APIs are written in JSON (some in XML). JSON:API is also a REST API Framework.

In webapplications REST-APIs are nearly everytime works with http request so i just dont know how client os software calls it. Http Request is surely one option, but i dont know if its the best.
Copy linkTweet thisAlerts:
@sibertDec 05.2021 — > @Change#1640184 Http Request is surely one option, but I don't know if it's the best.

Searching for the "best" is a never ending loop. There are always "better" options. :-)

I am testing AJAX and REST API created in Go using Postgresql as a database. (work in progress).

The advantage of API is that you isolate the query stuff in one place.

The advantage of AJAX is that you reduce flickering when updating an existing page.

The advantage of REST (and JSON as output) is that it is practically a standard way.

The advantage of Go is that it is almost as fast as C++ but simpler.

The advantage of Postgresql is the lack of Oracle. And the ACID compliance.

https://crud.go4webdev.org/api2
×

Success!

Help @Change spread the word by sharing this article on Twitter...

Tweet This
Sign in
Forgot password?
Sign in with TwitchSign in with GithubCreate Account
about: ({
version: 0.1.9 BETA 4.25,
whats_new: community page,
up_next: more Davinci•003 tasks,
coming_soon: events calendar,
social: @webDeveloperHQ
});

legal: ({
terms: of use,
privacy: policy
});
changelog: (
version: 0.1.9,
notes: added community page

version: 0.1.8,
notes: added Davinci•003

version: 0.1.7,
notes: upvote answers to bounties

version: 0.1.6,
notes: article editor refresh
)...
recent_tips: (
tipper: @Yussuf4331,
tipped: article
amount: 1000 SATS,

tipper: @darkwebsites540,
tipped: article
amount: 10 SATS,

tipper: @Samric24,
tipped: article
amount: 1000 SATS,
)...