Apache HTTPD

1. 2015.07.23
   1. introduction
      1. he Apache HTTP Server Project is an effort to develop and maintain an open-source HTTP server for modern operating systems including UNIX and Windows NT. The goal of this project is to provide a secure, efficient and extensible server that provides HTTP services in sync with the current HTTP standards.
   2. installation
      1. prerequisite
         1. libtool
         2. autoconf
         3. APR
         1. http://mungeol-heo.blogspot.kr/2015/07/apache-apr_29.html
         4. pcre
         5. gcc
         6. ntp
         7. perl
      2. wget http://mirror.apache-kr.org//httpd/httpd-2.4.16.tar.gz
      3. wget http://www.apache.org/dist/httpd/httpd-2.4.16.tar.gz.md5
      4. md5sum -c httpd-2.4.16.tar.gz.md5
      5. gzip -d httpd-2.4.16.tar.gz
      6. tar xvf httpd-2.4.16.tar
      7. cd httpd-2.4.16
      8. ./configure --with-included-apr
      9. make
      10. make install
      11. /usr/local/apache2/bin/apachectl -k start
      12. http://hostname
      13. /usr/local/apache2/bin/apachectl -k stop
      14. /usr/local/apache2/bin/apachectl -f /usr/local/apache2/conf/httpd.conf
   3. reference
      1. http://httpd.apache.org/

Apache APR

1. 2015.07.23
   1. introduction
      1. The mission of the Apache Portable Runtime (APR) project is to create and maintain software libraries that provide a predictable and consistent interface to underlying platform-specific implementations. The primary goal is to provide an API to which software developers may code and be assured of predictable if not identical behaviour regardless of the platform on which their software is built, relieving them of the need to code special-case conditions to work around or take advantage of platform-specific deficiencies or features.
   2. installation
      1. APR
         1. wget http://apache.tt.co.kr//apr/apr-1.5.2.tar.gz
         2. wget http://www.apache.org/dist/apr/apr-1.5.2.tar.gz.md5
         3. md5sum -c apr-1.5.2.tar.gz.md5
         4. gzip -d apr-1.5.2.tar.gz
         5. tar xvf apr-1.5.2.tar
         6. ln -s apr-1.5.2 apr
      2. APR-Util
         1. wget http://apache.tt.co.kr//apr/apr-util-1.5.4.tar.gz
         2. wget http://www.apache.org/dist/apr/apr-util-1.5.4.tar.gz.md5
         3. md5sum -c apr-util-1.5.4.tar.gz.md5
         4. gzip -d apr-util-1.5.4.tar.gz
         5. tar xvf apr-util-1.5.4.tar
         6. ln -s apr-util-1.5.4 apr-util
   3. reference
      1. https://apr.apache.org/

Hive - "Execution failed with exit status: 3"

note: solutions for resolving "Execution failed with exit status: 3" error while using "hive.auto.convert.join=true" setting.

1. use "hive.auto.convert.join=false" or
2. reduce "hive.smalltable.filesize" or
3. increase maximum size for the Java process heap memory which will be passed to Java -Xmx. (If you happen to use cloudera manger, then it will be at 'hive -> configuration -> gateway default group -> resource management -> client java heap size in bytes')

I suggest to use option 3, if it is possible for your specific use case.

LogStash -> RabbitMQ Server -> ES 순서로 indexing 하기

• LogStash에서 output로 elasticsearch-river 사용
• 상세 내용: http://logstash.net/docs/1.4.2/outputs/elasticsearch_river
• RabbitMQ 서버 및 management 설치
• 상세내용: http://mungeol-heo.blogspot.kr/2015/01/rabbitmq.html
• elasticsearch-river-rabbitmq plugin 설치
• 상세내용: http://mungeol-heo.blogspot.kr/2015/01/elasticsearch-river-rabbitmq.html
• LogStash 실행 시 RabbitMQ와의 authentication 실패 에러 발생
• LogStash에서 기본으로 guest 계정으로 RabbitMQ와 통신하도록 설정
• RabbitMQ 새 버전에서 guest 계정은 localhost에서만 접속 가능
• 새 계정 생성
• sudo rabbitmqctl add_user test test
• sudo rabbitmqctl set_user_tags test administrator
• sudo rabbitmqctl set_permissions -p / test ".*" ".*" ".*"
• RabbitMQ Management에 접속 테스트
• x.x.x.x:15672
• id:test
• pwd: test
• 접속 성공
• LogStash 설정에서 test 계정으로 RabbitMQ와 통신하도록 수정
• 통신 성공
• test 용 apache 로그를 indexing
• indexing 성공

java.lang.IllegalArgumentException: Invalid format: "XXX" is malformed at "XXX"

1. an example error

'java.lang.IllegalArgumentException: Invalid format:
"30/Jan/2015:15:01:03" is malformed at "Jan/2015:15:01:03"' error

2. check the LANG of system

echo $LANG

3. change it if it is not "en_US.UTF-8"

- For instance, at CentOS
- vi /etc/sysconfig/i18n 
- LANG="en_US.UTF-8"
- restart the system

Security setting - Failed to start services on hadoop, such as name node.

Note: this is the one of the reasons why the services of hadoop fail to start.

1. The security setting of linux system might the reason why your services of hadoop fail to start while using, for example, CDH.
2. Some time, you have to apply "chmod" to commands, because of your security setting of linux system.
3. for instance,

chmod 755 df netstat dmesg find ps c++ g++ last make top w who whoami x86_64-redhat-linux-* ...

4. And, the commands you have to change are depends on your security setting.
5. As a example, remember to delete name node directory before restart name node. Otherwise it will keep failing.

JRI vs. Rserve vs. Rsession

	JRI	Rserve	Rsession
Documentation	부족 (중)	부족 (상)	부족 (하)
API	부족 (중)	부족 (상)	부족 (하)
Installation	쉬움 (상)	보통 (하)	보통 (중)
Configuration	쉬움 (상)	보통 (하)	보통 (중)
Programming	쉬움 (상)	보통 (하)	보통 (중)
Community	부족 (중)	부족 (상)	없음 (하)
Server/Client		O	O
Windows	O		O