Nikto2

I have been working on a project for my information security class. It requires me to test and gather information on a server before attempting to penetrate it. I have managed to build a good list of information on the server, but I have not managed to penetrate it yet. Of course I'm trying to do this without the use of scripts or applications designed for this server's vulnerabilities, so I'm doing it the hard way, but honestly, did anyone expect anything less of me?

So I'm working though trying to find all the tools I can use to discover all the possible vulnerabilities and I remember nikto. For those who are not familiar with Nikto, it is a web server vulnerability tool, a very vertically aligned form of metasploit (which I wish had student licenses). Nikto 2 has come along way since the last time I looked at it and seems to be very stable. The thing I like most about Nikto is the mutation capability, being able to change what I need to accomplish my goal. This goes beyond just adding parameter tags, to being able to actively get content loaded on the server. It also has a export to metasploit function which enables this to be added to a pen tester's suite of tools. Nice.

Within a few minutes and a good nmap scan I was able to determine a mostly complete range of vulnerabilities on the project server. Of course the hard part is actually utilizing these vulnerabilities and exploiting them, but then again, that what I'm being graded on. Nikto 2 is working flawlessly on my ubuntu server, my Solaris VM, and my OSX laptop (10.7 Lion).

Apache2 and Personal Web Browsing in Leopard

The answer to Friday's post is simple. Apple installed Apache 2, which has a different directory structure than that of apache.

The answer can be found on Apple's Forums. Also PHP is turned off by default, so be sure to edit the httpd.conf file to turn it back on.

From the post:
"I got PHP working with Leopard by modifying the httpd.conf file that you can get to by going to Go -> Go To Folder, /etc then going into the apache2 folder and copying httpd.conf to the Desktop (it won't let you edit in place). Find the line that says LoadModule php5_module etc...... and remove the # from the start. Save the file and drag it back into the apache2 folder, you'll have to authenticate to get it in there. Then restart apache by switching personal web sharing off and on in the sharing pref pane.

I had the same problem with my personal web sharing folder for my username not working on both the machines I installed it on (as an upgrade). The machines web sharing is working tho, just not the one for each user account. To fix it, create a file called shortusername.conf (where shortusername is your shortusername, eg, joebloggs and in it put this...

Directory "/Users/shortusername/Sites/"
Options Indexes MultiViews
AllowOverride None
Order allow,deny
Allow from all
/Directory

Save the file and put it into the apache2/users folder - restart personal web sharing and boom!, its working now."

I did all this in the terminal, sudo of course.

I hope this helps.

Apache post Leopard 10.5 install

The upgrade itself went fine. However, now I cannot see in a browser:
http://localhost/~ChrisCopeland

http://localhost works fine

I checked out 2-3 other leopard upgrades - it's consistent on all machines.

My (current) permissions are:
drwxrwxrwx+ 14 ChrisCopeland ChrisCopeland 476 Oct 26 14:20 Sites

Apple tech support thought I was on server. The dude was not aware that apache comes on all installs. Once we got over that hurdle he was still a little confused. I looked through the httpd.conf file but could not find anything out of the ordinary.

Can anyone else verify? Got any ideas?

Setup is:
Model Name: Mac Pro
Model Identifier: MacPro1,1
Processor Name: Dual-Core Intel Xeon
Processor Speed: 2.66 GHz
Number Of Processors: 2
Total Number Of Cores: 4
L2 Cache (per processor): 4 MB
Memory: 2 GB
OS X 10.5

How To: The Urchin Data Extractor (u5data_extractor)

You can get the perl scripts for customizing Urchin data at the Google Urchin Support Page. I read the little documentation on this subject, which is a basic how to, without much resource. Urchin support firms charge something serious to get this kind of thing done, and here I am being a nice guy, giving away what I learned FOR FREE.

So let's begin with the lessons I learned.

1. Use some form of linux/unix. I could not, for the life of me, get any of these scripts to work with Windows and I think this is because of the path. The perl script is looking for a unix like path. I'm sure there are those people out there, smarter than I, who can get this to work on a windows server, but I am not one of them. The examples I give will be run from a Macintosh running OS X 10.4.10, ActiveState Perl, and the bash. In addition I would like to thank the wonderful folks (yet again) over at macosxhints forums as well as unix.com forums for helping me get my syntax correct in my scripts.

2. Use a step by step process.

3. Verify your data, and backup! The last thing you want to do is run an untested and "use at your own risk" script on your Urchin reports.

4. Do not always believe the available documentation.

5. When report testing, use small segments of data for your report. It saves time and you get to test your text scrubber faster.

Ok - now let's get to the logical process. What I wanted to do was to pull certain reports from Urchin and post them to a database, preferably some flavor of SQL.

The process will look something like this.
1. run perl script with start date, end date, report type, and number of items returned.
2. save report as a text file
3. scrub text file for bad characters, bad lines, and data which is not applicable.
4. comma delimit the file
5. hand csv file to sql import engine.

sounds easy right? It is for the most part.

The u5data_extractor script will do a lot of this work for you. This is the usage section of the script, which will also show up in the command line if you call the script with ~$ perl u5data_extractor. I removed the copyright and some other text for the purpose of posting to the blog.
###########################################################
# Usage: u5data_extractor.pl [--begin YYYYMMDD] [--end YYYYMMDD] [--help]
# [--language LA] [--max N] [--profile PROFILE]
# [--report RRRR] [--urchinpath PATH]
#
# Where:
# '--begin YYYYMMDD' specifies the starting date (default: one week ago)
# '--end YYYYMMDD' specifies the ending date (default: yesterday)
# '--help' displays this message
# '--language LA' specifies the language for the report. Available
# languages are: ch, en, fr, ge, it, ja, ko, po, sp, and sw
# '--max N' is the maximum number of entries printed in the top 10 report
# types (default is 10).
# '--profile PROFILE' specifies the profile to retrieve data from. The
# default is specified at the beginning of this script
# '--report RRRR is the 4-digit number for the report (default is 1102)
# Run this script with --help to see a list of available reports
# '--urchinpath PATH' specifies the path to the Urchin distribution.
# Note that you can edit the script and set your path as a default
###################################################

Giving the script your default path:
You will need to give the script the path to the Urchin Directory.
this is the line for my machine (following a unix path):
my $urchinpath = "/usr/local/urchin"; # Path to the Urchin distribution

Give the script your default profile:
You will need to give the script the default profile.
This is the line for a made up profile in the script.
my $profile = "My Default Profile"; # Name of the default profile
This is important - you do not have to use %20 to represent spaces if you are using the quotes. Urchin, by default, stores the profile directories with %20 for whitespace characters.

The report number is a difficult thing. Where do you find those reports? I found an article, somewhere, which shows the report numbers. Have no fear, I made a list for you of the urchin report numbers.

I will give an example, since none was really given for me. Let's say I want to run a report from Jan 01, 2007 to Jan 27, 2007 for the report "Visitors & Sessions"
so when you call the script, you will be using the following syntax:
perl u5data_extractor --begin 20070101 --end 20071027 --report 1903 --max 10

this will generate the output to the standard out (screen), which I will not post due to privacy reasons.

If you want to redirect the output feel free to do so
perl u5data_extractor --begin 20070101 --end 20071027 --report 1903 --max 10>>output.file

Tomorrow I will post my scrubbing process as well as the script I used to call backup the data and generate the reports.

Enjoy!

More Fun with Lynx

I grew up using gopher servers before there was a www or http, so when the real "web" came along it was needless to say awesome. One of the first web browsers I used was Lynx.

Lynx is a very very simple browser, very useful in scripts and for checking to see how a search engine views the webpage. If lynx cannot see your content, it is very doubtful that a seach-bot will see it too.

So the last post shows how to use lynx to call Google's caching times. This will show you how you can automate lynx to do automatic retrieval of web information for you.

Here is a simple script which will read a file in line by line and pass the information off to lynx for a Google search.

#!/bin/bash
cat ${1} | while read mySearchTerm; do
lynx -source -accept_all_cookies "http://www.google.com/search?q=$mySearchTerm"
done

This script will throw everything to the standard out. What I do is pass this information on to a text file or to grep for counting purposes.

#!/bin/bash
cat ${1} | while read mySearchTerm; do
lynx -source -accept_all_cookies "http://www.google.com/search?q=$mySearchTerm" |grep -c 'pattern.to.count'>> /path/to/text/file.txt
done

and now we have auto document retreival from Google. A word of warning, because this will take whatever is in the line, you must be careful of non-alpha numeric characters like !@#$%^&*-\/, as these will be passed on to Google too, which can alter the search results. You can also use things like the 'date' command or other small *nix programs to alter the url fed to lynx. If you want to time this sort of script you can always use crontab functionality found in unix, linux, os x. Be sure to read up on the man page for lynx.

Enjoy.