Menu
Community /Pin to ProfileBookmark
@NogDogJul 06.2019 — #First: put all that data into a database, and run that database in its own server (not the web server).
Second: if you ever reach a state where you actually have so much data that one database server is not enough in terms of storage size (and these days that's a _lot_ of data), search on "database sharding" as an option. If it's not storage that's the issue, but database performance due to high user load (likely from multiple web servers behind your load-balancer(s)), then maybe search on "database read-only replicas".@NogDogJul 08.2019 — #"Replication" in the context I was describing has nothing to do with the uniqueness of the data. It has to do with the entire database being replicated across 1 or more other database servers so that database reads can be load-balanced across them much the same as HTTP requests can be load-balanced across multiple web servers. In read-heavy contexts (which is typical for most web apps), one DB server will be the target of all database write requests, which will then be propagated by the replication system to the read-only databases. Read requests are then load-balanced to the read-only replicas to spread the load. @NogDogJul 08.2019 — #@coleio#1606014
It can be done in the application code by hitting a different database host for write requests than it does for read-only requests, or it can be done by a load-balancing tool that the application uses as it's "database host", which then decides which actual DB host to use depending on the type of query being sent.@NogDogJul 09.2019 — #
It may depend on the DBMS you choose, and what tools are available for it. If you can let some sort of smart middleware handle it, then you don't have to worry about it in the application code itself. The app I work on uses PostgreSQL on an Amazon RDS DB service, which handles the master/replication database piece, but we decide in the application code whether to hit the master or the replicant (basically only sending our most-used read queries to the replicant, and sending everything else to the master).
More than one
This is simply a hypothetical problem. Let’s say I have a web server and it stores general user information (PW, email, age etc…). Now let’s say that the harddrive is full to the brim with this data and I need to expand to a second server so new users can sign up. What tools do I need to achieve this?
Sign in
to post a comment12 Comments(s) ↴
0
@NogDogJul 06.2019 — #First: put all that data into a database, and run that database in its own server (not the web server).Second: if you ever reach a state where you actually have so much data that one database server is not enough in terms of storage size (and these days that's a _lot_ of data), search on "database sharding" as an option. If it's not storage that's the issue, but database performance due to high user load (likely from multiple web servers behind your load-balancer(s)), then maybe search on "database read-only replicas".
0
@NogDogJul 08.2019 — #0
@NogDogJul 08.2019 — #It can be done in the application code by hitting a different database host for write requests than it does for read-only requests, or it can be done by a load-balancing tool that the application uses as it's "database host", which then decides which actual DB host to use depending on the type of query being sent.
0
@NogDogJul 09.2019 — #>@coleio#1606022 which method is more suitable for a large scale application?
It may depend on the DBMS you choose, and what tools are available for it. If you can let some sort of smart middleware handle it, then you don't have to worry about it in the application code itself. The app I work on uses PostgreSQL on an Amazon RDS DB service, which handles the master/replication database piece, but we decide in the application code whether to hit the master or the replicant (basically only sending our most-used read queries to the replicant, and sending everything else to the master).