Results 1 to 11 of 11

Thread: Database options for (very) large databases

  1. #1
    Join Date
    Aug 2007

    Question Database options for (very) large databases

    I was just wondering if anyone could recommend a server platform for use with very large databases. I'm currently rather well versed with MySQL but if anyone knows of any problems with it's performance when it comes to large database sizes then could they let me know? I seem to remember that both Google and Youtube use MySQL don't they?

  2. #2
    Join Date
    Jan 2009
    I would look at the SQL Server 2008 clustered on a Windows HPC server.

    Hope this helps,

    Larry Darrah
    MS Architect Evangelist

  3. #3
    Join Date
    Aug 2007
    Thanks. I'd rather not use anything Windows based to be honest, I have a strong aversion to anything to do with M$.
    What about MySQL's handling of a table of say, 10'000 rows? Anyone?

  4. #4
    Join Date
    Nov 2002
    Baltimore, Maryland
    Oracle seems to work well and work well with big chunks of data. And SQL Server does have its limitations size-wise.
    “The power of the Web is in its universality. Access by everyone regardless of disability is an essential aspect.”
    —Tim Berners-Lee, W3C Director and inventor of the World Wide Web

  5. #5
    Join Date
    Jan 2007
    MySQL should perform quite nicely. It's the DBMS of choice for most web apps, big and small. And most of the caveats that apply to MySQL will apply to any other DBMS as well. I recommend taking a look at High Performance MySQL 2nd ed..
    Jon Wire

    thepointless.com | rounded corner generator

    I agree with Apple. Flash is just terrible.

    Use CODE tags!

  6. #6
    Join Date
    Aug 2005
    The Garden State
    define "large database" how many concurrent users? how large is the data? how many schemas?
    Acceptable Use | SQL Forum FAQ | celery is tasteless | twitter

    celery is tasteless - currently needing some UI time

  7. #7
    Join Date
    Aug 2007
    Thanks for the replies, I will definitely take a look at that book, I'll either order a copy or cycle down to my library. As far as the database size goes, it'll be about 10'000 rows, as far as data size, it'll mostly be quite small, probably about five or six columns, with mostly short text (categories, names), there might be one big column of text, but even so that cell won't contain more than about 300 words.
    I'm not sure about concurrent users. It shouldn't be more than say, 500 but that could be a severe underestimate. At this stage I'm not sure.

  8. #8
    Join Date
    Feb 2003
    That is nowhere near what I'd call large. MySQL will handle it fine

  9. #9
    Join Date
    Aug 2007
    Initially I thought it'd be bigger, so I decided to cover my bases, as it were.
    Since I am going to use MySQL, what can I use to search it? I ask this because I'd like to use Sphinx or Lucene (preferably Sphinx) but both use persistent processes (which I gather a lot of hosts dislike). I could use MySQL's built in full-text search but it isn't as quick and efficient, or so I've been led to believe...
    Any advice?

  10. #10
    Join Date
    Jan 2007
    If you get a VPS or dedicated server you don't have to worry about what your hosting provider thinks about running additional daemons. Sphinx is supposed to do a really great job with fulltext indexing. And it's flexible too. I'd grab myself a VPS (at least) and use that in conjunction with MySQL.

    Though for 10,000 rows (or even 100,000 or 1,000,000), I think MySQL's fulltext search would do fine. I think it's mostly a question of flexibility in search features at the scale you're dealing with.
    Jon Wire

    thepointless.com | rounded corner generator

    I agree with Apple. Flash is just terrible.

    Use CODE tags!

  11. #11
    Join Date
    Aug 2007
    This is true, I could write my own search functions and add functionality to the MySQL full-text searching but I think it'd get slow if I did that and someone's already done it, in the form of Sphinx, so I'm tempted to just go with that anyway. Thanks for the advice. I think I'll get a VPS, I dedicated server is unnecessary at this point.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
HTML5 Development Center



X vBulletin 4.2.2 Debug Information

  • Page Generation 0.11048 seconds
  • Memory Usage 2,956KB
  • Queries Executed 15 (?)
More Information
Template Usage (32):
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_global_above_footer
  • (1)ad_global_below_navbar
  • (1)ad_global_header1
  • (1)ad_global_header2
  • (1)ad_navbar_below
  • (1)ad_showthread_firstpost_sig
  • (1)ad_showthread_firstpost_start
  • (1)ad_thread_first_post_content
  • (1)ad_thread_last_post_content
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)headinclude_bottom
  • (11)memberaction_dropdown
  • (1)navbar
  • (4)navbar_link
  • (1)navbar_moderation
  • (1)navbar_noticebit
  • (1)navbar_tabs
  • (2)option
  • (11)postbit
  • (11)postbit_onlinestatus
  • (11)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available (6):
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files (26):
  • ./showthread.php
  • ./global.php
  • ./includes/class_bootstrap.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/functions_navigation.php
  • ./includes/class_friendly_url.php
  • ./includes/class_hook.php
  • ./includes/class_bootstrap_framework.php
  • ./vb/vb.php
  • ./vb/phrase.php
  • ./includes/functions_facebook.php
  • ./includes/functions_calendar.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_notice.php
  • ./packages/vbattach/attach.php
  • ./vb/types.php
  • ./vb/cache.php
  • ./vb/cache/db.php
  • ./vb/cache/observer/db.php
  • ./vb/cache/observer.php 

Hooks Called (73):
  • init_startup
  • friendlyurl_resolve_class
  • init_startup_session_setup_start
  • database_pre_fetch_array
  • database_post_fetch_array
  • init_startup_session_setup_complete
  • global_bootstrap_init_start
  • global_bootstrap_init_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • load_show_variables
  • load_forum_show_variables
  • global_state_check
  • global_bootstrap_complete
  • global_start
  • style_fetch
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • strip_bbcode
  • friendlyurl_clean_fragment
  • friendlyurl_geturl
  • forumjump
  • cache_templates
  • cache_templates_process
  • template_register_var
  • template_render_output
  • fetch_template_start
  • fetch_template_complete
  • parse_templates
  • fetch_musername
  • notices_check_start
  • notices_noticebit
  • process_templates_complete
  • friendlyurl_redirect_canonical
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • bbcode_parse_start
  • postbit_imicons
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • memberaction_dropdown
  • tag_fetchbit
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • build_navigation_data
  • build_navigation_array
  • check_navigation_permission
  • process_navigation_links_start
  • process_navigation_links_complete
  • set_navigation_menu_element
  • build_navigation_menudata
  • build_navigation_listdata
  • build_navigation_list
  • set_navigation_tab_main
  • set_navigation_tab_fallback
  • navigation_tab_complete
  • fb_like_button
  • showthread_complete
  • page_templates