-
automation question
Everyone,
So, I have been thinking recently about automation. This morning I
listened to a talk by one of the three dbas at Youtube (from the MySQL
Users Conference last year). Think about that. They mentioned
100,000,000 pageviews in one day (the data was from 2006). And THREE
dbas. Of course this is done by automating anything you can. This
isn't the first time I have heard of such ratios of servers/dbas.
While I don't have the servers or the traffic that they do at Youtube we
are growing quite nicely and adding db servers on a fairly regular
basis. It is time to put some serious thought into automating some
things. I was wondering what experience others have had with any
technologies (I am modestly familier with "expect" and have touched
"dsh").
Thoughts??
thanks,
keith
--
B. Keith Murphy
Paragon Consulting Services
http://www.paragon-cs.com
850-637-3877
-
RE: automation question
3 ideas come to mind-
cron and or cruisecontrol for off-hours scriptingant for handling cross-platform scriptingmaven for implementing dependency checking as well as the ability to pull from online repositories
Youtube.comI worked with a company last summer that implemented identical functionality and wanted to firewall youtube videos to use their (considerably more capable as far as supporting every codec on the planet) player..youtube said if you link/use/href any amount of content to our site then you MUST use our player..its no WONDER they have that incredible pageview traffic ..
Is there some specific functionality you want to Automate???Thanks/Martin Gainty______________________________________________Disclaimer and confidentiality noteEverything in this e-mail and any attachments relates to the official business of Sender. This transmission is of a confidential nature andSender does not endorse distribution to any party other than intended recipient. Sender does not necessarily endorse content contained within this transmission.> Date: Thu, 3 Jan 2008 13:06:07 -0500> From: bmurphy@paragon-cs..com> To: mysql@lists.mysql.com> Subject: automation question> > Everyone,>> So, I have been thinking recently about automation. This morning I > listened to a talk by one of the three dbas at Youtube (from the MySQL > UsersConference last year). Think about that. They mentioned > 100,000,000 pageviews in one day (the data was from 2006). And THREE > dbas. Of course thisis done by automating anything you can. This > isn't the first time I haveheard of such ratios of servers/dbas. > > While I don't have the servers or the traffic that they do at Youtube we > are growing quite nicely and adding db servers on a fairly regular > basis. It is time to put some serious thought into automating some > things. I was wondering what experience others have had with any > technologies (I am modestly familier with "expect" and have touched > "dsh"). > > Thoughts??> > thanks,> > keith> > > -- > B. Keith Murphy> > Paragon Consulting Services> http://www.paragon-cs.com> 850-637-3877> > > -- > MySQL General Mailing List> For list archives: http://lists.mysql.com/mysql> To unsubscribe: http://lists.mysql.com/mysql?unsub=mgainty@hotmail.com>
_________________________________________________________________
The best games are on Xbox 360. Click here for a special offer on an Xbox 360 Console.
http://www.xbox.com/en-US/hardware/wheretobuy/
-
Re: automation question
Martin,
Martin Gainty wrote:
> 3 ideas come to mind-
> cron and or cruisecontrol for off-hours scripting
> ant for handling cross-platform scripting
> maven for implementing dependency checking as well as the ability to
> pull from online repositories
>
I will take a look at these. I am certainly familiar with cron. The
others not so much.
> Youtube.com
> I worked with a company last summer that implemented identical
> functionality and wanted to firewall youtube videos to use their
> (considerably more capable as far as supporting every codec on the
> planet) player..youtube said if you link/use/href any amount of
> content to our site then you MUST use our player..its no WONDER they
> have that incredible pageview traffic ..
>
> Is there some specific functionality you want to Automate???
I have some things in mind, but basically any task that I am performing
across multiple servers I want to automate. While I might have time to
manually do things across 20 servers (not really... but that is the
current state) I won't be able to do so across 100. Make sense?
> Thanks/
> Martin Gainty
>
> ______________________________________________
> Disclaimer and confidentiality note
> Everything in this e-mail and any attachments relates to the official
> business of Sender. This transmission is of a confidential nature and
> Sender does not endorse distribution to any party other than intended
> recipient. Sender does not necessarily endorse content contained
> within this transmission.
>
>
> > Date: Thu, 3 Jan 2008 13:06:07 -0500
> > From: bmurphy@paragon-cs.com
> > To: mysql@lists.mysql.com
> > Subject: automation question
> >
> > Everyone,
> >
> > So, I have been thinking recently about automation. This morning I
> > listened to a talk by one of the three dbas at Youtube (from the MySQL
> > Users Conference last year). Think about that. They mentioned
> > 100,000,000 pageviews in one day (the data was from 2006). And THREE
> > dbas. Of course this is done by automating anything you can. This
> > isn't the first time I have heard of such ratios of servers/dbas.
> >
> > While I don't have the servers or the traffic that they do at
> Youtube we
> > are growing quite nicely and adding db servers on a fairly regular
> > basis. It is time to put some serious thought into automating some
> > things. I was wondering what experience others have had with any
> > technologies (I am modestly familier with "expect" and have touched
> > "dsh").
> >
> > Thoughts??
> >
> > thanks,
> >
> > keith
> >
> >
> > --
> > B. Keith Murphy
> >
> > Paragon Consulting Services
> > http://www.paragon-cs.com
> > 850-637-3877
> >
> >
> > --
> > MySQL General Mailing List
> > For list archives: http://lists.mysql.com/mysql
> > To unsubscribe: http://lists.mysql.com/mysql?unsub=mgainty@hotmail.com
> >
>
>
> ------------------------------------------------------------------------
> The best games are on Xbox 360. Click here for a special offer on an
> Xbox 360 Console. Get it now!
>
--
B. Keith Murphy
Paragon Consulting Services
http://www.paragon-cs.com
850-637-3877
-
Re: automation question
Martin,
I took a quick look at the tools that you mentioned. Seems you are
heavily involved in Java :)
We don't use Java at all, and other than using cron (which I do use for
backups and standard stuff like that) not really what I am looking for.
After thinking about this, I suspect that this will all just be a custom
scripting with something like dsh for the distributed part. Not really
looking for source code control -- more like things to manage the
distribution of standardized my.cnf files (which we already control
using subversion) to 100 servers at a time. Looking to scale efforts so
I don't have to manually do things.
thanks,
Keith
B. Keith Murphy wrote:
> Martin,
>
> Martin Gainty wrote:
>> 3 ideas come to mind-
>> cron and or cruisecontrol for off-hours scripting
>> ant for handling cross-platform scripting
>> maven for implementing dependency checking as well as the ability to
>> pull from online repositories
>>
> I will take a look at these. I am certainly familiar with cron. The
> others not so much.
>> Youtube.com
>> I worked with a company last summer that implemented identical
>> functionality and wanted to firewall youtube videos to use their
>> (considerably more capable as far as supporting every codec on the
>> planet) player..youtube said if you link/use/href any amount of
>> content to our site then you MUST use our player..its no WONDER they
>> have that incredible pageview traffic ..
>>
>> Is there some specific functionality you want to Automate???
> I have some things in mind, but basically any task that I am
> performing across multiple servers I want to automate. While I might
> have time to manually do things across 20 servers (not really... but
> that is the current state) I won't be able to do so across 100. Make
> sense?
>
>> Thanks/
>> Martin Gainty
>>
>> ______________________________________________
>> Disclaimer and confidentiality note
>> Everything in this e-mail and any attachments relates to the official
>> business of Sender. This transmission is of a confidential nature and
>> Sender does not endorse distribution to any party other than intended
>> recipient. Sender does not necessarily endorse content contained
>> within this transmission.
>>
>>
>> > Date: Thu, 3 Jan 2008 13:06:07 -0500
>> > From: bmurphy@paragon-cs.com
>> > To: mysql@lists.mysql.com
>> > Subject: automation question
>> >
>> > Everyone,
>> >
>> > So, I have been thinking recently about automation. This morning I
>> > listened to a talk by one of the three dbas at Youtube (from the MySQL
>> > Users Conference last year). Think about that. They mentioned
>> > 100,000,000 pageviews in one day (the data was from 2006). And THREE
>> > dbas. Of course this is done by automating anything you can. This
>> > isn't the first time I have heard of such ratios of servers/dbas.
>> >
>> > While I don't have the servers or the traffic that they do at
>> Youtube we
>> > are growing quite nicely and adding db servers on a fairly regular
>> > basis. It is time to put some serious thought into automating some
>> > things. I was wondering what experience others have had with any
>> > technologies (I am modestly familier with "expect" and have touched
>> > "dsh").
>> >
>> > Thoughts??
>> >
>> > thanks,
>> >
>> > keith
>> >
>> >
>> > --
>> > B. Keith Murphy
>> >
>> > Paragon Consulting Services
>> > http://www.paragon-cs.com
>> > 850-637-3877
>> >
>> >
>> > --
>> > MySQL General Mailing List
>> > For list archives: http://lists.mysql.com/mysql
>> > To unsubscribe: http://lists.mysql.com/mysql?unsub=mgainty@hotmail.com
>> >
>>
>>
>> ------------------------------------------------------------------------
>> The best games are on Xbox 360. Click here for a special offer on an
>> Xbox 360 Console. Get it now!
>>
>
>
--
B. Keith Murphy
Paragon Consulting Services
http://www.paragon-cs.com
850-637-3877
-
Re: automation question
learnt.
On Jan 4, 2008 10:15 AM, B. Keith Murphy wrote:
> Martin,
>
> I took a quick look at the tools that you mentioned. Seems you are
> heavily involved in Java :)
>
> We don't use Java at all, and other than using cron (which I do use for
> backups and standard stuff like that) not really what I am looking for.
> After thinking about this, I suspect that this will all just be a custom
> scripting with something like dsh for the distributed part. Not really
> looking for source code control -- more like things to manage the
> distribution of standardized my.cnf files (which we already control
> using subversion) to 100 servers at a time. Looking to scale efforts so
> I don't have to manually do things.
>
> thanks,
>
> Keith
>
> B. Keith Murphy wrote:
> > Martin,
> >
> > Martin Gainty wrote:
> >> 3 ideas come to mind-
> >> cron and or cruisecontrol for off-hours scripting
> >> ant for handling cross-platform scripting
> >> maven for implementing dependency checking as well as the ability to
> >> pull from online repositories
> >>
> > I will take a look at these. I am certainly familiar with cron. The
> > others not so much.
> >> Youtube.com
> >> I worked with a company last summer that implemented identical
> >> functionality and wanted to firewall youtube videos to use their
> >> (considerably more capable as far as supporting every codec on the
> >> planet) player..youtube said if you link/use/href any amount of
> >> content to our site then you MUST use our player..its no WONDER they
> >> have that incredible pageview traffic ..
> >>
> >> Is there some specific functionality you want to Automate???
> > I have some things in mind, but basically any task that I am
> > performing across multiple servers I want to automate. While I might
> > have time to manually do things across 20 servers (not really... but
> > that is the current state) I won't be able to do so across 100. Make
> > sense?
> >
> >> Thanks/
> >> Martin Gainty
> >>
> >> ______________________________________________
> >> Disclaimer and confidentiality note
> >> Everything in this e-mail and any attachments relates to the official
> >> business of Sender. This transmission is of a confidential nature and
> >> Sender does not endorse distribution to any party other than intended
> >> recipient. Sender does not necessarily endorse content contained
> >> within this transmission.
> >>
> >>
> >> > Date: Thu, 3 Jan 2008 13:06:07 -0500
> >> > From: bmurphy@paragon-cs.com
> >> > To: mysql@lists.mysql.com
> >> > Subject: automation question
> >> >
> >> > Everyone,
> >> >
> >> > So, I have been thinking recently about automation. This morning I
> >> > listened to a talk by one of the three dbas at Youtube (from the
> MySQL
> >> > Users Conference last year). Think about that. They mentioned
> >> > 100,000,000 pageviews in one day (the data was from 2006). And THREE
> >> > dbas. Of course this is done by automating anything you can. This
> >> > isn't the first time I have heard of such ratios of servers/dbas.
> >> >
> >> > While I don't have the servers or the traffic that they do at
> >> Youtube we
> >> > are growing quite nicely and adding db servers on a fairly regular
> >> > basis. It is time to put some serious thought into automating some
> >> > things. I was wondering what experience others have had with any
> >> > technologies (I am modestly familier with "expect" and have touched
> >> > "dsh").
> >> >
> >> > Thoughts??
> >> >
> >> > thanks,
> >> >
> >> > keith
> >> >
> >> >
> >> > --
> >> > B. Keith Murphy
> >> >
> >> > Paragon Consulting Services
> >> > http://www.paragon-cs.com
> >> > 850-637-3877
> >> >
> >> >
> >> > --
> >> > MySQL General Mailing List
> >> > For list archives: http://lists.mysql.com/mysql
> >> > To unsubscribe:
> http://lists.mysql.com/mysql?unsub=mgainty@hotmail.com
> >> >
> >>
> >>
> >>
> ------------------------------------------------------------------------
> >> The best games are on Xbox 360. Click here for a special offer on an
> >> Xbox 360 Console. Get it now!
> >>
> >
> >
>
>
> --
> B. Keith Murphy
>
> Paragon Consulting Services
> http://www.paragon-cs.com
> 850-637-3877
>
>
> --
> MySQL General Mailing List
> For list archives: http://lists.mysql.com/mysql
> To unsubscribe:
> http://lists.mysql.com/mysql?unsub=y...0608@gmail.com
>
>
--
I'm a mysql DBA in china.
More about me just visit here:
http://yueliangdao0608.cublog.cn