Skip to main content

Harjeet Singh's Library tagged config   View Popular

18 Oct 07

Howtos - Whitepapers

How To: Connect to a remote mysql server using mysqlcc and ssh tunneling part 2 | Debian/Ubuntu Tips & Tricks

  • We saw last time, that in order to securely connect to a remote MySQL server running both mysql and ssh, we had to create a tunnel between our desktop and the server where we had to forward port 3307 on our desktop to port 3306 running locally on the remote server by giving the following intruction to ~/.ssh/config:



    Localforward 3307 localhost:3306




    This time, we want to forward port 3307 on our desktop to port 3306 on the MySQL server (let says it as 192.168.0.3 as IP adress). The only change we have to make to the previous configuration is to change the Localforward instruction by:



    Localforward 3307 192.168.0.3:3306




    And simply use the same mysqlcc configuration as the one given in the How To Connect to a remote mysql server using mysqlcc and ssh tunneling Tutorial. It is as simple as that :). Here is an overview of the final configuration we have deployed:


    .


    People which do not want to use ~/.ssh/config might use the following command:



    tester@laptop:~$ssh -L 3307:192.168.0.3:3306 myuser@remotesshserver.com




    Now, you can play around with port forwarding, later on, I will show you how to go even further and just do some useless be geeky thing.

How To Connect to a remote mysql server using mysqlcc and ssh tunneling. | Debian/Ubuntu Tips & Tricks

  • First of all, we are going to configure ssh. Edit or Create file ~/.ssh/config. Add the following lines (depending on your settings):


    Host myhostnickname


    Hostname myremotehost.com


    User mysshuser Localforward 3307 localhost:3306


    #only if using rsa or dsa key authentification


    IdentityFile ~/.ssh/myremotehost_id_dsa



    connect to your remote host


    $ssh myhostnickname

28 Sep 07

Webficient: How to Configure Swiftiply with Nginx

  • How to Configure Swiftiply with Nginx




    I've been testing the performance of various architectures for running Ruby Web applications and thought I'd share some reference information specific to one type of configuration, as publicly available documentation is sparse at the time of writing.

    Swiftiply is a proxying solution which boosts the performance of Ruby on Rails and Merb apps. One configuration allows you to use Swiftiply as an HTTP proxy, leveraging what are known as "Swiftiplied" Mongrels. These patched Mongrels become clients of the Swiftiply server and maintain a persistent connection, which is a reversal of the usual proxying solution (think Pound, Squid, etc.).

    Nginx is a lightweight, fast HTTP server, which is gaining popularity.

    I've setup Nginx as a reverse proxy to Swiftiply, which in turn has a direct connection to multiple Mongrel processes.

    So when an HTTP request comes in, the pipeline looks like this...

    HTTP request -> Nginx -> Swiftiply -> Mongrels

    To successfully make Nginx and Swiftiply play nicely with one another, the configuration is fairly straightforward.

    In my nginx.conf, the cluster consists of ONE server entry:
    upstream mongrel_test_cluster {
    server 127.0.0.1:4000;
    }

    This may appear strange at first glance but really, what you're telling Nginx is to proxy all requests to the same port being listened on by Swiftiply, which in turn, communicates with the Mongrels.

    Then, here are the relevant settings in swiftiply.yml:
    cluster_address: 127.0.0.1
    cluster_port: 4000
    daemonize: true
    epoll: true
    epoll_descriptors: 8192
    map:
    - incoming: localhost
    outgoing: 127.0.0.1:5000
    default: true
    docroot: /usr/local/httpd/testapp
    redeployable: true

    Swiftiply is listening on port 4000 and passing requests to port 5000 (to the "Swiftiplied" Mongrels).

    To transform your Mongrel into a Swiftiplied Mongrel, just add
    require 'swiftcore/swiftiplied_mongrel'

    to your application's configuration file.

    Then, it's as simple as firing up your Swiftiplied Mongrels and the Swiftiply server (both are covered in more detail in the Swiftcore docs), and Nginx.

    The other Swiftiply configuration, using "Evented" Mongrels, is much simpler and does not use the Swiftiply server.

    Just add this include to your Web application's configuration file:
    require 'swiftcore/swiftiplied_mongrel'

    and configure Nginx as you would with a standard set of Mongrels.
    upstream mongrel_test_cluster {
    server 127.0.0.1:4000;
    server 127.0.0.1:4001;
    server 127.0.0.1:4002;
    server 127.0.0.1:4003;
    }

    Then start up your Mongrels as described in the Swiftiply docs, and you're done.
25 Sep 07

How many Mongrel instances should I run?

    • How Many Mongrel Instances Should I Run?




      There is no set number that is “best” since that depends on factors like the
      type of application, server hardware, how dynamic the appication is, etc.




      I’ve found that 8-12 mongrel processes per CPU is about right, but I determined
      this by starting with 1 and then doing the following:




      Baseline Your Server




      Pick a URL to a small file that is running on your apache server and is not
      served by Mongrel at all. This URL will be your “best possible baseline”.




      Build your baseline measurement first. Using httperf, measure the speed of
      your URL so that you know how fast you could possibly get if you served
      everything static in ideal situations.




      Make sure you do this on a different machine over an ideal network.
      Not your damn wifi over a phone line through sixteen poorly configured routers.
      Right next to the box your testing with a fast switch and only one hop is the
      best test situation. This removes network latency from your test as a
      confounding factor.




      Baseline Rails and Mongrel




      Pick a page that’s a good representative page for your application. Make sure
      you disable logins to make this test easier to run. Hit this Rails page and
      compare it to your baseline page.




      • If your rails measurement is faster than your baseline
        measurement then you screwed up. Rails shouldn’t be faster than a file
        off your static server. Check your config.
      • If your rails measurement is horribly slow compared to baseline
        then you’ve got some config to do before you even start tuning the
        number of process. Repeat this test until one mongrel is as fast as
        possible.
      • Run the test and find out that one mongrel can support a --rate of 120 req/second.
      • Add another mongrel and run the test again with --rate 240. It handles this
        just find so you add another and get --rate 360.
      • Try another one and it dies. Giving --rate 480 gets you only a rate of 100.
        Your server has hit it’s max and broke.
      • Try tuning the --rate down at this and see if it’s totally busted (like, 4 mongrels
        only gets you --rate 380) or if it’s pretty close to 480.
      • That should do it. A good practice is to also look at the CPUs on the
        server with top and see what kind of thrashing you give the server.
      • httperf --server www.theserver.com --port 80 --uri /tested --num-conns <10 second count>
      • httperf --server www.theserver.com --port 80 --uri /tested --num-conns <10 second count> --rate <reported req/sec>



      Where <10 second count> is enough connections to make the test run for 10
      seconds. Start off with like 100 and keep raising it until it goes for 10
      seconds.




      Where <reported req/sec> is whatever httperf said the estimated
      requests/second were. What you’re doing here is seeing if it really can handle
      that much concurrency. Try raising it up and dropping it down to see the
      impact of performance on higher loads.

22 Sep 07

Cool Solutions: What is the "umask"? How can I set it?

    • The umask defines the permissions a new file will get - or better:
      the permissions it will not get.



      You can display the current umask numeric and as text:



      user@host:~ $ umask
      0027
      user@host:~ $ umask -S
      u=rwx,g=rx,o=


      The numbers mean the following:



      0 0 2 7
      | | | '--> permissions for others (o)
      | | '--> permissions for the group (g)
      | '--> permissions for the owner (user, u)
      '--> special permissions (SUID, SGID, sticky) - always 0 in umask


      The digits for user, group and others are the sum of:


      • 1 - execute permission (x)
      • 2 - write permission (w)
      • 4 - read permission (r)
      • all permissions for the file owner (user)
      • no write permissions (but read and execute permissions) for the group
      • no permissions for others


      You can specify the umask with the command umask 0027. The
      number can vary, of course. The umask you define this way is valid in
      the current shell and all child processes. If you set the umask in
      ~/.profile, it is valid for the whole time you are logged in.
      If you define it in a xterm, it is only valid for everything you do in
      this xterm.



      If you want to define the umask for a specific directory (example:
      group write permissions for a directory you use together with your
      colleges), you'll become sweating when using the umask command
      because it is always valid for all directories.



      The solution of this problem is setting a default ACL. The
      following command ensures that all new files in /home/shared/
      have all permissions (including write permissions) set for the group:



      setfacl -d -m mask:007 /home/shared/


      You should also set the sgid-bit for the directory and choose the
      wanted group using chgrp:



      chgrp the_team /home/shared/
      chmod g+s /home/shared/


      If /home/shared/ already contains subdirectories, you have
      to change their permissions as well. Tip: all mentioned commands know
      the -R option.



      Starting with KDE 3.5 (which will be contained in the upcoming SUSE
      Linux 10.1) you can easily define ACLs using the file properties dialog.



      If you want to access /home/shared/ only using samba, you
      can instead use the directory mask and create mask for
      the share (be warned: samba doesn't use the inverted permission mask as
      umask does!). You should also set the force group option.



      Original URL (german):


      http://suse-linux-faq.koehntopp.de/q/q-filesystems-umask.html
21 Sep 07

MySql: Give Root User Logon Permission From Any Host :: the How-To Geek

  • MySql: Give Root User Logon Permission From Any Host




    <script type="text/javascript"><!--
    google_ad_client = "pub-9173525300015284";
    google_ad_width = 300;
    google_ad_height = 250;
    google_ad_format = "300x250_as";
    google_ad_type = "text";
    google_ad_channel = "1086003014+9201668985+1966353953";
    google_color_border = "FFFFFF";
    google_color_bg = "FFFFFF";
    google_color_link = "003371";
    google_color_text = "000000";
    google_color_url = "003371";
    //--></script>
    <script src="http://pagead2.googlesyndication.com/pagead/show_ads.js" type="text/javascript"></script>








    <!-- google_ad_section_start -->

    Note that this is Not very secure, and should only be used for a local development box where you don't feel like setting up individual permissions, but still need to connect from other machines.


    To configure this feature, you'll need to update the mysql user table to allow access from any remote host, using the % wildcard.


    Open the command-line mysql client on the server using the root account.


    mysql -uroot



    Then you will want to run the following two commands, to see what the root user host is set to already:


    use mysql;
    select host, user from user;



    Here's an example of the output on my database, which is pretty much the default settings. Note that ubuntuserv is the hostname of my server.


    mysql> use mysql;
    Reading table information for completion of table and column names
    You can turn off this feature to get a quicker startup with -A
    Database changed


    mysql> select host,user from user;
    +—————+——————+
    | host | user |
    +—————+——————+
    | ubuntuserv | root |
    | localhost | debian-sys-maint |
    | localhost | root |
    +—————+——————+
    3 rows in set (0.00 sec)



    Now I'll update the ubuntuserv host to use the wildcard, and then issue the command to reload the privilege tables. If you are running this command, substitute the hostname of your box for ubuntuserv.


    update user set host='%' where user='root' and host='ubuntuserv';
    flush privileges;



    That's all there is to it. Now I was able to connect to that server from any other machine on my network, using the root account.

    Again, note this isn't very secure, and you should at least make sure that you've set a root password.


    <!-- google_ad_section_end -->



    The Geek is the founder of How-To Geek and a geek enthusiast. When he's not coming up with great how-to articles, he's probably writing at his personal blog.


20 Sep 07

MySQL AB :: MySQL 5.0 Reference Manual :: 2.4.16.3 Securing the Initial MySQL Accounts

    • Anonymous Account Password
      Assignment



      To assign passwords to the anonymous accounts, connect to the
      server as root and then use either
      SET PASSWORD or UPDATE. In
      either case, be sure to encrypt the password using the
      PASSWORD() function.



      To use SET PASSWORD on Windows, do this:


      shell> mysql -u root
      mysql> SET PASSWORD FOR ''@'localhost' = PASSWORD('newpwd');
      mysql> SET PASSWORD FOR ''@'%' = PASSWORD('newpwd');


      To use SET PASSWORD on Unix, do this:


      shell> mysql -u root
      mysql> SET PASSWORD FOR ''@'localhost' = PASSWORD('newpwd');
      mysql> SET PASSWORD FOR ''@'host_name' = PASSWORD('newpwd');


      In the second SET PASSWORD statement, replace
      host_name with the name of the server
      host. This is the name that is specified in the
      Host column of the
      non-localhost record for
      root in the user table. If
      you don't know what hostname this is, issue the following
      statement before using SET PASSWORD:


      mysql> SELECT Host, User FROM mysql.user;


      Look for the record that has root in the
      User column and something other than
      localhost in the Host
      column. Then use that Host value in the
      second SET PASSWORD statement.



      Anonymous Account Removal



      If you prefer to remove the anonymous accounts instead, do so as
      follows:


      shell> mysql -u root
      mysql> DROP USER '';


      The DROP statement applies both to Windows
      and to Unix. On Windows, if you want to remove only the
      anonymous account that has the same privileges as
      root, do this instead:


      shell> mysql -u root
      mysql> DROP USER ''@'localhost';


      That account allows anonymous access but has full privileges, so
      removing it improves security.



      root Account Password
      Assignment



      You can assign passwords to the root accounts
      in several ways. The following discussion demonstrates three
      methods:



      • Use the SET PASSWORD statement


      • Use the mysqladmin command-line client
        program


      • Use the UPDATE statement



      To assign passwords using SET PASSWORD,
      connect to the server as root and issue two
      SET PASSWORD statements. Be sure to encrypt
      the password using the PASSWORD() function.



      For Windows, do this:


      shell> mysql -u root
      mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd');
      mysql> SET PASSWORD FOR 'root'@'%' = PASSWORD('newpwd');


      For Unix, do this:


      shell> mysql -u root
      mysql> SET PASSWORD FOR 'root'@'localhost' = PASSWORD('newpwd');
      mysql> SET PASSWORD FOR 'root'@'host_name' = PASSWORD('newpwd');
15 Sep 07

Apache: Creating A Session-Aware Loadbalancer Using mod_proxy_balancer (Debian Etch) | HowtoForge - Linux Howtos and Tutorials

  • Apache: Creating A Session-Aware Loadbalancer Using mod_proxy_balancer (Debian Etch)



    <!-- start main content -->



    <script type="text/javascript">
    <!--
    document.write('<div style="float: right; margin: 0 0 10px 10px;">');
    //-->
    </script>

    <!-- BEGIN NetShelter Ad Tag for HowtoForge 120x600,160x600 -->
    <script language="JavaScript" type="text/javascript">
    if (!window.netshel_ord) { netshel_ord=Math.random()*10000000000000000;
    }
    if (!window.netshel_tile) { netshel_tile=1; }
    document.write('<script src="http://ad.doubleclick.net/adj/ns.howtoforge/howtos;sz=120x600,160x600;tile='+netshel_tile+';ord=' + netshel_ord + '?" language="JavaScript" type="text/javascript">');
    netshel_tile++;
    </script><script src="http://ad.doubleclick.net/adj/ns.howtoforge/howtos;sz=120x600,160x600;tile=2;ord=391161676682785.25?" language="JavaScript" type="text/javascript"></script>
    <script src="http://a.tribalfusion.com/j.ad?site=HowToForgecom&adSpace=ROS&size=120x600&noAd=1&requestID=20641551710.7027781348096246" language="javascript"></script>


    <!-- END AD TAG --><script type="text/javascript">
    <!--
    document.write('<div><a href="http://www.howtoforge.com/membership"><img src="http://www.howtoforge.com/themes/htf_glass/images/remove_ads.gif" border="0" alt="Remove ads"></a></div>');
    //-->
    </script>
    Remove ads
    <script type="text/javascript">
    <!--
    document.write('</div>');
    //-->
    </script>

    <!-- begin content -->



    Submitted by falko (Contact Author) (Forums) on Tue, 2007-05-29 16:42. :: Debian | Apache


    Apache: Creating A Session-Aware Loadbalancer Using mod_proxy_balancer (Debian Etch)


    Version 1.0

    Author: Falko Timme <ft [at] falkotimme [dot] com>

    Last edited 05/26/2007


    Since Apache 2.1, a new module called mod_proxy_balancer is available which lets you turn a system that has Apache installed into a loadbalancer. This loadbalancer retrieves requested pages from two or more backend webservers and delivers them to the user's computer. Users get the impression that they deal with just one server (the loadbalancer) when in fact there are multiple systems behind the loadbalancer that process the users' requests. By using a loadbalancer, you can lower the load average on your webservers. One important feature of mod_proxy_balancer is that it can keep track of sessions which means that a single user always deals with the same backend webserver. Most websites are database-driven nowadays with user logins etc., and you'd get weird results if a user logs in on one backend webserver, and then his next request goes to another backend webserver, meaning he'd get logged out again. You can avoid this by using mod_proxy_balancer's session-awareness.


    I do not issue any guarantee that this will work for you!

Apache Tomcat 6.0 - Manager App HOW-TO

    • Configuring Manager Application Access



      The description below uses the variable name $CATALINA_HOME
      to refer to the directory into which you have installed Tomcat 6,
      and is the base directory against which most relative paths are
      resolved. However, if you have configured Tomcat 6 for multiple
      instances by setting a CATALINA_BASE directory, you should use
      $CATALINA_BASE instead of $CATALINA_HOME for each of these
      references.




      It would be quite unsafe to ship Tomcat with default settings that allowed
      anyone on the Internet to execute the Manager application on your server.
      Therefore, the Manager application is shipped with the requirement that anyone
      who attempts to use it must authenticate themselves, using a username and
      password that have the role manager associated with them.
      Further, there is no username in the default users file
      ($CATALINA_HOME/conf/tomcat-users.xml) that is assigned this
      role. Therefore, access to the Manager application is completely disabled
      by default.



      To enable access to the Manager web application, you must either create
      a new username/password combination and associate the role name
      manager with it, or add the manager role
      to some existing username/password combination. Exactly where this is done
      depends on which Realm implementation you are using:


      • MemoryRealm - If you have not customized your
        $CATALINA_HOME/conf/server.xml to select a different one,
        Tomcat 6 defaults to an XML-format file stored at
        $CATALINA_HOME/conf/tomcat-users.xml, which can be
        edited with any text editor. This file contains an XML
        <user> for each individual user, which might
        look something like this:
        <user name="craigmcc" password="secret" roles="standard,manager" />

        which defines the username and password used by this individual to
        log on, and the role names he or she is associated with. You can
        add the manager role to the comma-delimited
        roles attribute for one or more existing users, and/or
        create new users with that assigned role.
      • JDBCRealm - Your user and role information is stored in
        a database accessed via JDBC. Add the manager role
        to one or more existing users, and/or create one or more new users
        with this role assigned, following the standard procedures for your
        environment.
      • JNDIRealm - Your user and role information is stored in
        a directory server accessed via LDAP. Add the manager
        role to one or more existing users, and/or create one or more new users
        with this role assigned, following the standard procedures for your
        environment.


      The first time you attempt to issue one of the Manager commands
      described in the next section, you will be challenged to log on using
      BASIC authentication. The username and password you enter do not matter,
      as long as they identify a valid user in the users database who possesses
      the role manager.



      In addition to the password restrictions the manager web application
      could be restricted by the remote IP address or host by adding a
      RemoteAddrValve or RemoteHostValve. Here is
      an example of restricting access to the localhost by IP address:

      <Context path="/manager" privileged="true"
      docBase="/usr/local/kinetic/tomcat6/server/webapps/manager">
      <Valve className="org.apache.catalina.valves.RemoteAddrValve"
      allow="127\.0\.0\.1"/>
      </Context>

Configuring & Using Apache Tomcat 6

    • Turn on Servlet Reloading


      The next step is to tell Tomcat to check the modification dates of
      the class files of requested servlets, and reload ones
      that have changed since they were loaded into the server's memory.
      This slightly degrades performance in deployment situations,
      so is turned off by default. However, if you fail to turn it on for
      your development server, you'll have to restart the server
      every time you recompile a servlet that has already been loaded into
      the server's memory. Since this tutorial discusses the
      use of Tomcat for development, this change is strongly recommended.


      To turn on servlet reloading, edit Edit
      install_dir/conf/context.xml and change


        <Context>


      to


        <Context reloadable="true" privileged="true">


      Note that the privileged entry is really to support the invoker
      servlet (see the following section), so you can omit that entry if you do not
      use the invoker.


      You can also:




      6. Turn on Directory Listings (Optional)



      In previous Tomcat versions, if you entered a URL ending in a slash
      (/) and there was no welcome-file in the directory (or servlet-mapping that matched the URL), Tomcat displayed a directory listing. In Tomcat 6, the default
      was changed from true to false for these directory listings. Many developers find
      it convenient to turn directory listings back on.
      To make this change, edit install_dir/conf/web.xml
      and change the init-param value of listings for the default servlet,
      as below. Do not confuse this
      Apache Tomcat-specific web.xml file with the standard
      one that goes in the WEB-INF directory of each Web application.


          <servlet>
      <servlet-name>default</servlet-name>
      <servlet-class>org.apache.catalina.servlets.DefaultServlet</servlet-class>
      <init-param>
      <param-name>debug</param-name>
      <param-value>0</param-value>
      </init-param>
      <init-param>
      <param-name>listings</param-name>
      <param-value>true</param-value>
      </init-param>
      <load-on-startup>1</load-on-startup>
      </servlet>


      You can also:

14 Sep 07

Search smarter with Apache Solr, Part 2: Solr for the enterprise

  • Search smarter with Apache Solr, Part 2: Solr for the enterprise

    Administration, configuration, and performance

    developerWorks
    Document options

    Document options requiring JavaScript are not displayed

    <script language="JavaScript" type="text/javascript">

    <!--
    document.write('<tr valign="top"><td width="8"><img src="http://www.ibm.com/i/c.gif" width="8" height="1" alt=""/></td><td width="16"><img alt="Set printer orientation to landscape mode" height="16" src="http://www.ibm.com/i/v14/icons/printer.gif" width="16" vspace="3" /></td><td width="122"><p><b><a class="smallplainlink" href="http://www.ibm.com/javascript:print()">Print this page</a></b></p></td></tr>');
    //-->
    </script>
    Set printer orientation to landscape mode

    Print this page


    <script language="JavaScript" type="text/javascript">
    <!-- 5.6 10/24 llk: added cdata around the subdirectory path of email gif-->
    <!--
    document.write('<tr valign="top"><td width="8"><img src="http://www.ibm.com/i/c.gif" width="8" height="1" alt=""/></td><td width="16"><img src="http://www.ibm.com/i/v14/icons/em.gif" height="16" width="16" vspace="3" alt="Email this page" /></td><td width="122"><p><a class="smallplainlink" href="http://www.ibm.com/javascript:void newWindow()"><b>E-mail this page</b></a></p></td></tr>');
    //-->
    </script>
    Email this page

    E-mail this page

    Sample code

    <!--START RESERVED FOR FUTURE USE INCLUDE FILES--><!-- 08/27/07 refreshed by jill, per MOC -->


    Watch these demos

    Integrate new tools and architectures into your environment -- fast!

    <!--END RESERVED FOR FUTURE USE INCLUDE FILES-->
    Rate this page

    Help us improve this content


    Level: Intermediate

    Grant Ingersoll (solr@grantingersoll.com), Senior software engineer, Center for Natural Language Processing at Syracuse University

    05 Jun 2007

    Lucene Java™ committer Grant Ingersoll rounds out his introduction to Solr with a survey of its features for the enterprise, including administration interfaces, advanced configuration options, and performance features such as caching, replication, and logging.
    <!--START RESERVED FOR FUTURE USE INCLUDE FILES--><script language="JavaScript" type="text/javascript">
    <!--
    if (document.referrer&&document.referrer!="") {
    // document.write(document.referrer);
    var q = document.referrer;
    var engine = q;
    var isG = engine.search(/google\.com/i);
    var searchTerms;
    //var searchTermsForDisplay;
    if (isG != -1) {
    var i = q.search(/q=/);
    var q2 = q.substring(i+2);
    var j = q2.search(/&/);
    j = (j == -1)?q2.length:j;
    searchTerms = q.substring(i+2,i+2+j);
    if (searchTerms.length != 0) {
    searchQuery(searchTerms);
    document.write("<div id=\"contents\"></div>");
    }
    }
    }
    //-->
    </script><!--END RESERVED FOR FUTURE USE INCLUDE FILES-->

    In Part 1 of this
    series, I
    introduced Apache Solr, an open source, HTTP-based search server that can
    be easily incorporated into a wide variety of Web applications. I
    demonstrated Solr's basic functionality, including indexing, searching, and
    browsing, and also introduced the Solr schema and explained its role in
    configuring Solr functionality. In this second half of the article, I complete my introduction to Solr by showcasing the features that make it a desirable solution for large-scale production environments. Topics covered include administration, caching, replication, and extensibility.

12 Sep 07

BlogFish: Scaling Rails with Apache 2.2, mod_proxy_balancer and Mongrel

  • Scaling Rails with Apache 2.2, mod_proxy_balancer and Mongrel



    Posted by Jonathan






    Unitl this week we used Lighttpd and FastCGI for MeinProf.de. The setup was nearly the same as described in the must read series scaling rails (1, 2, 3, 4) from poocs.net.




    We used this setup from day 1 but always had some small issues with Lighttpd. Lighttpd was crashing every couple of days. Nothing dramatic, we had a script that monitored Lighttpd and restarted it if necessary. During the last weeks Lighttpd started to crash once a day and lately even once an hour. This was unacceptable and as we knew that we were going to get some serious press coverage in Germany we looked for alternatives.


    43people and Basecamp use Apache 1.3 and FastCGI so this seemed like a good alternative. Just switch the webserver and we would be done. Unfortunately Apache 1.3 cannot loadbalance the FastCGI request and there is very little documentation on Apache 1.3 and remote FastCGI processes. Apache 2.0 is no better and has problems with mod_fastcgi. We needed remote FastCGI listeners as our hardware is quite old and we have many slow machines as opposed to a few fast ones that could use local FastCGI to handle the load.



    Enter Mongrel.



    Mongrel is a fast HTTP library and server for Ruby that is intended for hosting Ruby web applications of any kind using plain HTTP rather than FastCGI or SCGI. It is framework agnostic and already supports Ruby On Rails, Og+Nitro, and Camping frameworks.


    With Mongrel your application server becomes a webserver that speaks HTTP so you “only” need to loadbalance and proxy normal HTTP request to it. Mongrel was stable during our tests so we looked for the HTTP proxy solution. Apache had always mod_proxy and could therefore proxy HTTP requests but we needed to loadbalancer these. The are extra packages for this kind of stuff like Balance but we wanted something more integrated and didn’t want to introduce more components.


    Enter Apache 2.2 and mod_proxy_balancer.


    Apache 2.2 introduced a new proxy module, mod_proxy_balancer. This module does exactly this, it balances proxy requests. You can define a cluster of proxies and use this cluster in your mod_proxy statement instead of just one proxy server.




    With this setup we use Apache 2.2 to handle all incoming requests. Apache 2.2 uses mod_proxy to redirect the incoming HTTP requests to the mod_proxy_balancer cluster. The cluster consists of several Mongrel processes on each application server (and now also internal web server) and distributes the requests.

  • Looks cool enough, but how do I get Apache to serve the static stuff (images/css/js) but proxy the rest, without using a VirtualHost? Can't wrap it all in a Location block, because ProxyPass directives in a Location block ignore the first argument to ProxyPass (the url), and use the url in the Location block instead. So, I think I'd need to first use mod_alias to pull the images, stylesheets, and javascripts directories into Apache's namespace, then prevent those from being proxied, then proxy everything else.

    How does the performance of static content served by Apache compare to that served by Mongrel?

Chapter 16. Clustering

  • Configure Apache to load mod_jk

    Modify APACHE_HOME/conf/httpd.conf and add a single line at the end of the file:

    # Include mod_jk's specific configuration file  
    Include conf/mod-jk.conf

    Next, create a new file named APACHE_HOME/conf/mod-jk.conf:

    # Load mod_jk module
    # Specify the filename of the mod_jk lib
    LoadModule jk_module modules/mod_jk.so

    # Where to find workers.properties
    JkWorkersFile conf/workers.properties

    # Where to put jk logs
    JkLogFile logs/mod_jk.log

    # Set the jk log level [debug/error/info]
    JkLogLevel info

    # Select the log format
    JkLogStampFormat "[%a %b %d %H:%M:%S %Y]"

    # JkOptions indicates to send SSK KEY SIZE
    JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories

    # JkRequestLogFormat
    JkRequestLogFormat "%w %V %T"

    # Mount your applications
    JkMount /application/* loadbalancer

    # You can use external file for mount points.
    # It will be checked for updates each 60 seconds.
    # The format of the file is: /url=worker
    # /examples/*=loadbalancer
    JkMountFile conf/uriworkermap.properties

    # Add shared memory.
    # This directive is present with 1.2.10 and
    # later versions of mod_jk, and is needed for
    # for load balancing to work properly
    JkShmFile logs/jk.shm

    # Add jkstatus for managing runtime data
    <Location /jkstatus/>
    JkMount status
    Order deny,allow
    Deny from all
    Allow from 127.0.0.1
    </Location>
1 - 14 of 14
Showing 20 items per page

Highlighter, Sticky notes, Tagging, Groups and Network: integrated suite dramatically boosting research productivity. Learn more »

Join Diigo