systemsresearch

GIT client installation pains :)

2011-06-17T09:51:00.001-07:00

Install gcc

yum install gcc
Install zlib-devel package

Noticed that zlib-devel was needed by the git install process when I first tried to run configure on the git distribution.

yum install zlib-devel
Install curl-devel

Needed if you want to use http/https/webdav access to git repositories.

yum install curl-devel
Build Git from Source

Once you have got your CentOS 5.2 system running, you can build git from the source tree.
Download git

wget http://kernel.org/pub/software/scm/git/git-1.6.0.4.tar.gz
Unpack the Git source tree

tar xvzf git-1.6.0.4.tar.gz
Create a build target path

You will need to have a build target directory path on your build machine that is the same as the path to your home directory on your target virtual host account. In this example we’re calling it
/home/clientname.

Replace clientname with the name of your home directory on hostgatore which is usually the same as your domainname.

This is where the build/install process will end up “installing” the compiled and other run time files:

mkdir /home/clientname
Run Configure on source tree

Change directory on the build machine to the top of the git source tree. (Where you unpacked the tar file git-1.6.0.4.tar.gz it does not have to be in /home/clientname)

cd git-1.6.0.4

Configure and set the prefix to the home directory that is your home directory on the target virtual host.

./configure --prefix=/home/clientname/git_root

Build the code and have it be installed in the directory that you defined in the –prefix line above
make
make install
Assuming all goes well, it will build the code in the directory

git-1.6.0.4

and install everything in the directory

/home/clientname/git_root

It will automatically create the git_root directory but you have to have created /home/clientname
Tar it up and scp it to target

cd /home/clientname
tar cvzf git_root.tgz git_root
scp git_root.tgz userid@clienturl:

Unpack it on target

ssh userid@clienturl
tar xvzf git_root.tgz

Set up environment to use the newly installed git

Add the path to the newly created bin to your path:

Edit your startup file such as ~/.bash_profile and add:

PATH=/home/clientname/git_root/bin:$PATH

You have to log out and log back in or source .bash_profile to update the path. You can also type that line into your current session at the prompt and have it immediately active.
Use Git as normal!

I found that I can now use Git as normal except that with Hostgator they seemed to be blocking the git protocol or something. I could not say:

git clone git://github.com/rberger/project.git

But I could say:

git clone git@github.com:rberger/project.git

(assuming I had my ssh keys setup to access github as git)

or

git clone http://github.com/rberger/project.git

Note that you do not say ./git. We are assuming that you added git to your path and you can just say git and it will find the executables via the $PATH

If you didn’t want to change $PATH, you could give a full path like:

/home/clientname/git_root/bin/git clone git@github.com:rberger/project.git
Save your CentOS 5.2 Amazon Image

If you want to, you can save the CentOS 5.2 customized image you created on Amazon EC2.

Ssh into the root account on your running CentOS amazon instance, copy your Amazon EC2 certificate and private key to the home directory of root on the running CentOS instance and run the following commands:
Build the new AMI

Replace cert-your_cert_sig.pem, your-private-key-sig and your-amazon-userid with the appropriate values for your EC2 certificate.

ec2-bundle-vol -d /mnt --cert cert-your_cert_sig.pem --privatekey pk-your-private-key-sig.pem -u your-amazon-userid -s 3096

The -s 3096 tells it to create an image that is 3GB big. You may have to make it bigger if the command fails. You can check that this is the problem by adding the –debug flag to the above command.
Stash the image into your S3 account

Use your own ws-access-key-id and aws-secret-access-key

ec2-upload-bundle -b centos-5.2-git -m /mnt/image.manifest.xml -a aws-access-key-id -s aws-secret-access-key
Git to your hearts content!

Here is the gzipped tar file of git built to run on hostgator. I don’t know if it will work in any other directory than the one it was built for though.

IPMI on Dell Poweredge running red hat

2010-03-29T06:07:00.001-07:00

http://lonesysadmin.net/2007/06/21/how-to-configure-ipmi-on-a-dell-poweredge-running-red-hat-enterprise-linux/

How to Configure IPMI on a Dell PowerEdge running Red Hat Enterprise Linux

By Bob Plankers on Jun 21, 2007 in Featured, How To, System Administration

This is intended to help fairly knowledgeable people get IPMI working on their hosts so they can issue remote commands to their hardware. I focus on Red Hat Enterprise Linux on a Dell, but it is likely to work on other hosts, distributions, and OSes, too. This works for me on Dell PowerEdge 1850, PowerEdge 2850, Dell PowerEdge 1950, and Dell PowerEdge 2950 hardware.

Dell PowerEdge 1650, 2650, and 1750 servers have an older implementation of IPMI which will let you issue commands locally, but not to these models over the network.

Before you begin:

The Baseboard Management Controller (BMC) is the thing that implements IPMI. It piggybacks on the first built-in NIC so you have to have that attached to the network on the hosts you wish to manage. It uses its own IP address (so you need an extra one).

If you are new to this get a server that is nearby to act as your test machine. Most of the IPMI commands that shut the host down also kill the BMC. If that happens you’ll need to go power the machine on manually. Until you figure out exactly what is okay and what isn’t you’ll be pushing the power button a lot.

You will need a Linux host to send the IPMI commands from. Undoubtedly there are ways to send these commands from other operating systems, but as I am a Linux guy I’ll use that as my example. Feel free to post comments addressing other OSes.

Getting the OS prepared:

1) Install IPMItool and the startup scripts. On Red Hat Enterprise Linux install the OpenIPMI, OpenIPMI-tools, OpenIPMI-libs, and OpenIPMI-devel packages. That will get you everything you need. There are similar packages available for other distributions (SuSE, Ubuntu, CentOS, etc.). You’ll need IPMItool on any machine you want to configure, and any machine you want to send commands from.

2) Enable the IPMI service:

/sbin/chkconfig ipmi on

3) Start the IPMI service, which will load the kernel modules for you:

/sbin/service ipmi start

Configure the BMC for Remote Usage:

1) There are two ways to configure the BMC. You can configure it through the boot-time menu (Ctrl-E), where you can set the management password and IP address information. Or, you can configure it with ipmitool from the OS. Replace my sample IP address, gateway, and netmask with your own:

/usr/bin/ipmitool -I open lan set 1 ipaddr 192.168.40.88
/usr/bin/ipmitool -I open lan set 1 defgw ipaddr 192.168.40.1
/usr/bin/ipmitool -I open lan set 1 netmask 255.255.255.0
/usr/bin/ipmitool -I open lan set 1 access on

2) Secure the BMC, so unauthorized people can’t power cycle your machines. To do this you want to change the default SNMP community, the “null” user password, and the root user password. First, set the SNMP community, either to a random string or something you know:

/usr/bin/ipmitool -I open lan set 1 snmp YOURSNMPCOMMUNITY

Then set the null user password to something random. Replace CRAPRANDOMSTRING with something random and secure:

/usr/bin/ipmitool -I open lan set 1 password CRAPRANDOMSTRING

Last, set the root user password to something you know:

/usr/bin/ipmitool -I open user set password 2 REMEMBERTHIS

Double-check your settings with:

/usr/bin/ipmitool -I open lan print 1

Trying it:

1) You can set an environment variable, IPMI_PASSWORD, with the password you used above. That will save some typing:

export IPMI_PASSWORD="REMEMBERTHIS"

If you use this substitute the “-a” in the following commands with a “-E”.

2) From another machine issue the following command, obviously replacing the IP with the target BMC’s IP:

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power status

You should get something like:

Chassis Power is on

If you get anything else, or nothing, double-check to make sure the BMC is set right, you entered the right password, and the IP it has is reachable from the machine you’re on. You can double-check your work via the Ctrl-E boot menu, too.

Beyond that, get familiar with:

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power off

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a chassis power cycle

/usr/bin/ipmitool -I lan -U root -H 192.168.40.88 -a sel list

For me, a “chassis power off” command kills the box. “SEL” is the system event log.

You can issue all of these commands locally, too:

/usr/bin/ipmitool sel list

Hopefully this helps a little. If you find any errors in this please leave me a comment or send me an email. Thanks!

permissions for rsa based authentication for ssh

2010-03-24T13:34:00.001-07:00

make sure that the *right* permissions are set up

drwx------ 2 virender virender 4096 May 7 04:51 .ssh
-rw------- 1 virender virender 393 May 7 03:41 .ssh/authorized_keys

Booting Citrix Vdisk target both as a physical machine as well as via a XenServer VM

2010-02-02T07:09:00.001-08:00

How to Create a Xen VM to Physical Common Image
Document ID: CTX119877 / Created On: Mar 27, 2009 / Updated On: Aug 12, 2009
Average Rating: 5 (1 ratings)
View products this document applies to link to products this document applies to

Summary

This article describes how to create a vDisk that boots both a physical (bare metal) end point and a XenServer Virtual Machine (VM).

Background

The essential components to create a common image are:

• A physical master target device

• XenServer Tools installed

• Provisioning Server target device installed

Because the image can be booted within a virtual machine, updates, backups, and modifications are streamlined and simplified.

Prerequisites

• Installation and configuration of Provisioning Server, referred to as the Provisioning Server Streaming Server

• Installation and configuration of XenServer on supported hardware

• Proper XenServer Platinum Licensing

• Creation and formatting of a Provisioning Server vDisk for the image-build process

• Ensure Pre-boot Execution Environment (PXE) is supported on the local network

• Ensure Dynamic Host Configuration Protocol (DHCP) is installed and configured on the local network

• Creation of a master target device on physical hardware with supported Windows operating system, fully patched and configured

• XenServer Tools installer

• Provisioning Server target device

Procedure

The procedure below is divided into three sections. Use the appropriate sections for your environment.

• Install a supported Windows operating system on a physical master target device with latest patches and device drivers.

• If running Windows XP or Windows Server 2003, go to the Modify Boot.ini - Windows XP/2003 section.

• If running Windows Vista or Windows Server 2008, go to the Modify…Boot Parameter Command - Windows Vista/2008 section.

Modify Boot.ini – Windows XP/2003

1. Log on to the master target device as a domain administrator, or a domain user (with local install privileges).

2. Right click on My Computer and select Properties.
The System Properties dialogue box appears.

3. On the Advanced tab, under Startup and Recovery, click Settings.

4. Under System startup, select Edit.
This opens the Boot.ini file in Notepad.

5. Append the switch /PV|=2000 to the end of the line that states the operating system in use.

6. Click File, and then save the changes made to the boot.ini.

7. Reboot the master target device.

8. Go to the Install the XenServer Tools section below.

Modify Boot Parameters – Windows Vista/2008

1. Log on to the master target device as a domain administrator, or a domain user (with local install privileges).

2. Open a command prompt.

3. Run the following command:
bcdedit /set loadoptions “PV|=2000”

4. Reboot the master target device.

Install the XenServer Tools

1. Log on to the master target device as a domain administrator, or a domain user (with local install privileges).

2. Install the windows-pvdrivers-xensetup.exe which can be found on the XenServer-4.1.0-install-cd.iso.

3. The Citrix XenServer Windows Tools Setup warning appears. Click Yes to continue the installation.

4. Follow the onscreen prompts and select the defaults. At the Choose Install Location dialogue box, click Install.

5. When prompted by Windows Plug and Play dialogs, select the option to find drivers automatically. When prompted, select Yes for any unsigned driver dialog.

6. When prompted, reboot the master target device.

Install the Provisioning Server Target Device Software

1. Log on to the master target device as a domain administrator, or a domain user (with local install privileges).

2. Install the Provisioning Server target device software.

3. Follow the onscreen prompts selecting all defaults.

4. Exit the reboot prompt without rebooting and continue to Deleting the PVS entry in the Boot.ini.

If running Windows XP or Windows Server 2003, go to the Modify Boot.ini - Windows XP/2003 section below.

If running Windows Vista or Windows Server 2008, go to the Modify… Boot Parameter Command - Windows Vista/2008 section below.

Modify (deleting PVS entry) in Boot.ini – Windows XP/2003

1. Log on to the master target device as a domain administrator, or a domain user (with local install privileges).

2. Right click on My Computer and select Properties.
The System Properties dialogue box appears.

3. On the Advanced tab, under Startup and Recovery, click Settings.

4. Under System startup, select Edit.
This opens the Boot.ini file in Notepad

5. Remove the switch /PV|=2000 to the end of the line that states the operating system in use.

6. Click File and then click Save to save the changes made to the boot.ini.

7. Shut down the master target device.

8. Go to the Image the Provisioning Server Master Target Device section below.

Modify (Deleting Provisioning Server entry) using boot parameters command – Windows Vista/2008

1. Log on to the master target device as a domain administrator, or a domain user (with local install privileges).

2. Open a command prompt.

3. Run the following command:
bcdedit /deletevalue loadoptions

4. Shut down the master target device.

5. For the next steps, go to the Image the Provisioning Server Master Target Device section below.

Image the Provisioning Server Master Target Device

1. Boot the master target device, enter the BIOS configuration utility, and configure the boot order to PXE boot first. Save configuration settings and continue the boot process. When PXE booting a master target device for the first time, the Provisioning Server Streaming Service prompts for the following information:
Client Name – Give the master target device a meaningful name
Description – Give the master target device a meaningful description
Select vDisk – Select the vDisk that was created and formatted for this installation
Boot From – Select boot from hard drive

2. When the master target device is successfully restarted, log on to the system as a domain administrator, or a domain user (with local install privileges).

3. Click Start, then Programs, then Citrix Provisioning Server and select Provisioning Server Image Builder.

4. Before building the image, click Optimize to optimize the device image.

5. Click Browse to select the vDisk to build the image to and click Build to start the image build.

6. Upon completion of the image build, shut down the master target device.

Configuration Steps – vDisk Administration

The following steps outline the details for setting the vDisk to Standard Image mode, which allows you to share the vDisk among multiple devices and virtual machines.

1. In the Provisioning Server console, highlight the vDisk icon in the object tree pane. Right click and select the Properties; the Disk Properties dialogue box opens. Click on the Disk Mode tab to access the Disk Access Mode properties. Using the drop down menu select Standard Image as the access mode. Once selected, click OK.

2. The master target device is currently set to boot from a hard drive. In the Provisioning Server Console, highlight the master target device icon in the object tree pane. Right click and select the Properties; the Client Properties dialogue box opens. In the Provisioning Server console, highlight the master target device icon in the object tree pane. Click on the Disk tab to access the client boot options. Using the drop down menu, select Virtual Disk first next to Boot Order. Once selected, click OK.

You should now be able to boot the vDisk in Standard Image Mode to both the physical machine and a XenServer VM.

perl module installed check

2010-01-21T06:29:00.001-08:00

Let's say that you want to know whether module Tie::Hash is installed. To find out, execute the following from the command line:

perl -MTie::Hash -e 1

If you don't get any output from the above command then the module is installed; if you get an error, it's not installed

RSA key generation for SSH linux

2010-01-12T09:08:00.001-08:00

If your daily activity requires loging in a lot of Linux systems through SSH, you will be happy to know (if you don't already) that there's a way to allow secure, authenticated remote access, file transfer, and command execution without having to remember passwords for each individual host you connect.

The $HOME/.ssh/authorized_keys file contains the RSA keys allowed for RSA authentication. Each line contains one key, which consists of the following fields: options, bits, exponent, modulus and comment. The first field is optional, bits, exponent and modulus fields give the RSA key and the last field isn't used at all in the authentication process, but it will be somewhat convenient to the user, for instance to know which key is for which machine.

Before we start, make sure your computer has a ssh client installed and the remote Linux system has ssh installed and sshd running, with RSA authentication enabled (RSAAuthentication yes in /etc/ssh/sshd_config).

First, you will need to generate the local RSA key:

# ssh-keygen -t rsa

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
(It's safe to press enter here, as the /root/.ssh is the default and recommended directory to hold the RSA file.)

Enter passphrase (empty for no passphrase):
Enter same passphrase again:
(The password you enter here will need to be entered every time you use the RSA key but fortunately, you can set NO passphrase by pressing Enter. However, the upside is that you only have to remember this one passphrase for all the systems you access via RSA authentication and you can change the passhrase later with "ssh-keygen -p".)

Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.

Once the public key has been generated, it's time to upload it on any Linux systems you usually log into. It's recommended you use scp as the file transfer utility:

# scp .ssh/id_rsa.pub username@hostname.com:~

This command will copy the id_rsa.pub file in the $HOME directory. For instance, if you used root as the username, the file will be found in the /root directory and if you used a normal user, the file will be in the /home/that.user/ directory.

Next, connect to the remote host through SSH, with the username you used in the step above. RSA authentication won't be available just yet, so you'll have to use the old method to login. Once you are connected, add the new hostkey to the file /root/.ssh/authorized_keys or /home/user/.ssh/authorized_keys. If the .ssh directory doesn't exist, create it.

# cd $HOME
# cat id_rsa.pub >> .ssh/authorized_keys

The two right-angles will add the contents of id_rsa.pub file to the authorized_keys file, so in case the file already exists, you won't have to worry about the existing content being modified.

You are all set. To test the RSA authentication, initiate a ssh connection from your PC to one of the Linux systems:

# ssh username@remote.hostname.com

If everything worked out well, you should be either asked for the passpharase (if you entered one), or get directly logged in. If you are prompted for the ssh password or get an error message, retry the above command using -v in order to turn verbose mode on and to be able to track down and correct the problem.

scp

2009-11-27T12:08:00.001-08:00

scp user@host:

i keep forgetting the syntax

Socket pooling

2009-11-05T09:27:00.000-08:00

http://articles.techrepublic.com.com/5100-22_11-5756029.html

(Listen up: This is material not included in our book. So if you have the book, print this out and stick it in the book! -Tom)

Do you sometimes feel that ISA Server was designed to drive you crazy? If so, when were those times? I bet you felt like you were going nuts when you tried to do one of the following things:

Publish an FTP Server running on the ISA Server itself
Publish a Web Site running on the ISA Server itself
Publish a NNTP server running on the ISA Server itself
Publish a SMTP server running on the ISA Server itself
Publish an Outlook Web Access site running on the ISA Server itself
Configuring ISA Server 2000 : Building Firewalls for Windows 2000
By Deb and Tom Shinder

Amazon.com
Notice something that all of these things have in common? That right! They all have the service in question running on the ISA Server itself. It makes you think they should have called it Server Punishing rather than Publishing.

What is maddening about this situation is that the Help File says that it's no problem to publish services running on the ISA Server. Just make a couple of changes to the listening IP address for the particular service, then publish the service using that IP address. And away you go! The problem is that it doesn't work.

The Misery of Socket Pooling

All of these problems are related to a little known issue with IIS 5.0. The current version of IIS uses something called Socket Pooling which, if I understand the issue correctly, putatively improves performance. While it may improve performance, it completely whacks your ability to publish services on the ISA Server itself.

What Socket Pooling does is allow a particular service to listen on all IP addresses on a particular computer. For example, if you have one internal IP address bound to the internal interface and two IP addresses bound to the external interface, then Socket Pooling will allow the service, such as the IIS 5.0 FTP Service, to listen on all three IP addresses.

Let's take a closer look at this example.

Open the Internet Information Services console from the Administrative Tools menu.

Right click on the default FTP site and click Properties.
In the Default FTP Site Properties dialog box you will see what appears below. Actually, you won't see what appears below until you click the down-arrow in the drop-down list box for TCP Port. Note that I have two IP addresses on this particular computer. I will select 192.168.1.186 with the intent of having the FTP service listen on that IP address only. After making the selection, click Apply and then OK.

Open a command prompt window. at the command prompt type netstat -na. You'll see something like what appears below. Notice the top entry that says:

TCP 0.0.0.0:21 0.0.0.0:0 LISTENING

What this means is that port 21 is listening on all IP addresses, not just 192.168.1.186. This happens in spite of our configuring the IIS FTP service to use only 192.168.1.186 for the FTP service. This is the "benefit" of socket pooling.

What do you think will happen if I have packet filtering enabled and try to publish the FTP service on 192.168.1.186? It won't work! I could enable a packet filter to allow inbound TCP 21, but then we're not publishing the server. Is just listening on the interface that is directly connected to the Internet, and if I tried to put the server, the publishing rule would still not work.

Fixing the Socket Pooling Problem
In order to get the IIS W3SVC, FTPSVC and NNTPSVC to play nice with Web and Server Publishing Rules, we have to disable Socket Pooling. To do this, perform the following steps:

Open a command prompt and navigate to the \Inetpub\Adminscripts\ folder
Type net stop msftpsvc and press [ENTER]
Type in the following command:

cscript adsutil.vbs set msftpsvc/disablesocketpooling true

and then press [ENTER]

You should see what appears below.

At the command prompt type net start msftpsvc and press [ENTER].
Now let's run netstat -na again. You should see what appears below.

Notice that TCP port 21 is now listening on 192.168.1.186 and is no longer listening on 0.0.0.0. No more socket pooling for Port 21! Now you can publish the FTP service on the internal interface (almost, you actually have to make a registry change too).
This same procedure will work for the WWW and NNTP services. Just replace msftpsvc with w3svc or nntpsvc. However, this procedure will not work with the SMTP service.

How to Disable SMTP Service Socket Pooling
To disable Socket Pooling for the SMTP service, you have to use a utility called Mdutil.exe. You can download this utility from the 'Tools' section of the 'Learning Zone'

Put the Mdutil.exe executable in the \Inetpub\Adminscripts folder.
Open a command prompt window and type the following command:
mdutil set -path smtpsvc/1 -value 1 -dtype 1 -prop 1029 -attrib 1

Go to the Internet Information Services console, right click on the SMTP service, and click Properties. Change the listening address to the internal interface of the ISA Server.
For more information on Socket Pooling and the SMTP service, check out:

http://msdn.microsoft.com/library/psdk/iisref/apro9zon.htm

Conclusion
IIS 5.0 Socket Pooling has caused a great deal pain for budding ISA Server administrators. It would have been nice to have this issue mentioned in the Help File. But then if they did that, I wouldn't have had a good reason to write this article.

After disabling Socket Pooling using the methods described, you will be able to publish services located on the ISA Server itself. Although it is always preferable to remove all unnecessary services on the ISA Server, sometimes you can't get around the problem because of financial limitations. In that case, you have to run extra services on the ISA Server.

A special note for you Outlook Web Access admins. This solution will fix the problem you have had with publishing Outlook Web Access when it is located on the ISA Server itself. If you are trying to do this, I highly recommend that you read Martin Grasdal's article - "OWA and ISA Server" on this subject.

DMZ

2009-11-05T09:23:00.000-08:00

http://articles.techrepublic.com.com/5100-22_11-5756029.html

The concept of the DMZ, like many other network security concepts, was borrowed from military terminology. Geopolitically, a demilitarized zone (DMZ) is an area that runs between two territories that are hostile to one another or two opposing forces' battle lines. The term was first widely used to refer to the strip of land that cuts across the Korean peninsula and separates the North from the South. In computer networking, the DMZ likewise provides a buffer zone that separates an internal network from the often hostile territory of the Internet. Sometimes it's called a "screened subnet" or a "perimeter network," but the purpose remains the same.

In this article, we'll look at how the DMZ works and different security architectures for building DMZs. In the second article of this two-part article, we'll talk about what computers should (and shouldn't) be placed in the DMZ and how to monitor DMZ activity.

How the DMZ Works
Unlike the geopolitical DMZ, a DMZ network is not a no-man's land that belongs to nobody. When you create a DMZ for your organization, it belongs to you and is under your control. However, it is an isolated network that's separate from your corporate LAN (the "internal" network). The DMZ uses IP addresses belonging to a different network ID.

If you think of the internal network as the "trusted" network and the external public network (the Internet) as the "untrusted" network, you can think of the DMZ as a "semi-trusted" area. It's not as secured as the LAN, but because it is behind a firewall, neither is it as non-secure as the Internet. You can also think of the DMZ as a "liaison network" that can communicate with both the Internet and the LAN while sitting between the two, as illustrated by Figure A.

Figure A

What does this accomplish? You can place computers that need to communicate directly with the Internet (public servers) in the DMZ instead of on your internal network. They will be protected by the outer firewall, although they are still at risk simply because they have direct contact with Internet computers. Because the DMZ is only "semi-secure," it's easier to hack a computer in the DMZ than on the internal network. The good news is that if a DMZ computer does get hacked, it doesn't compromise the security of the internal network, because it's on a completely separate, isolated network.

Why put any computers in this riskier network? Let's take an example: in order to do its job (make your Web site available to members of the public), your Web server has to be accessible to the Internet. But having a server on your network that's accessible from the Internet puts the entire network at risk. There are three ways to reduce that risk:

You could pay a hosting company to host your Web sites on their machines and network. However, this gives you less control over your Web servers.
You could host the public servers on the firewall computer. However, best security practices say the firewall computer should be dedicated solely to act as a firewall (this reduces the chances of the firewall being compromised), and practically speaking, this would impair the firewall's performance. Besides, if you have a firewall appliance running a proprietary OS, you won't be able to install other services on it.
The third solution is to put the public Web servers on a separate, isolated network: the DMZ.
Creating a DMZ Infrastructure
The DMZ is created by two basic components: IP addresses and firewalls. Remember that two important characteristics of the DMZ are:

It has a different network ID from the internal network
It is separated from both the Internet and the internal network by a firewall
IP Addressing Scheme
A DMZ can use either public or private IP addresses, depending on its architecture and firewall configuration. If you use public addresses, you'll usually need to subnet the IP address block that you have assigned to you by your ISP, so that you have two separate network IDs. One of the network IDs will be used for the external interface of your firewall and the other will be used for the DMZ network.

When you subnet your IP address block, you must configure your router to know how to get to the DMZ subnet.

You can create a DMZ within the same network ID that you use for your internal network, by using Virtual LAN (VLAN) tagging. This is a method of partitioning traffic that shares a common switch, by creating virtual local area networks as described in IEEE standard 802.1q. This specification creates a standard way of tagging Ethernet frames with information about VLAN membership.

If you use private IP addresses for the DMZ, you'll need a Network Address Translation (NAT) device to translate the private addresses to a public address at the Internet edge. Some firewalls provide address translation.

Whether to choose a NAT relationship or a routed relationship between the Internet and the DMZ depends on the applications you need to support, as some applications don't work well with NAT.

DMZ Firewalls
When we say that a firewall must separate the DMZ from both the internal LAN and the Internet, that doesn't necessarily mean you have to buy two firewalls. If you have a "three legged firewall" (one with at least three network interfaces), the same firewall can serve both functions. On the other hand, there are reasons you might want to use two separate firewalls (a front end and a back end firewall) to create the DMZ.

Figure A above illustrates a DMZ that uses two firewalls, called a back to back DMZ. An advantage of this configuration is that you can put a fast packet filtering firewall/router at the front end (the Internet edge) to increase performance of your public servers, and place a slower application layer filtering (ALF) firewall at the back end (next to the corporate LAN) to provide more protection to the internal network without negatively impacting performance for your public servers. Each firewall in this configuration has two interfaces. The front end firewall has an external interface to the Internet and an internal interface to the DMZ, whereas the backend firewall has an external interface to the DMZ and an internal interface to the corporate LAN.

When you use a single firewall to create a DMZ, it's called a trihomed DMZ. That's because the firewall computer or appliance has interfaces to three separate networks:

The internal interface to the trusted network (the internal LAN)
The external interface to the untrusted network (the public Internet)
The interface to the semi-trusted network (the DMZ)
The trihomed DMZ looks like Figure B.

Figure B

A trihomed DMZ uses a "three legged" firewall to create separate networks

Even if you use a single trihomed firewall to protect both the DMZ and the internal network, you should be able to configure separate rules for evaluating traffic depending on its origin and destination. That is, there should be separate rules for:

Incoming traffic from the Internet to the DMZ
Incoming traffic from the DMZ to the internal LAN
Incoming traffic from the Internet to the internal network
Outgoing traffic from the internal network to the DMZ
Outgoing traffic from the internal network to the Internet
Outgoing traffic from the DMZ to the Internet
The DMZ actually reduces the complexity of filtering traffic, because you can have one rule for all the computers in the DMZ. If you were hosting the public servers on the internal network, you would need to configure different rules for each hosting server, and you would have to "publish" each server to allow it to be accessed from the Internet.

You'll probably want to block traffic from the Internet to the internal computers. You should also restrict traffic from the DMZ to the internal network, as well as traffic from the Internet to the DMZ. Allow only the traffic that is necessary for your users to access the resources they need. This means using the "principle of least privilege" in that your default is to start by denying all traffic and then allowing protocols and opening ports on a "need to know" basis.

Vendor Support for DMZs
Major hardware and software vendors support the DMZ concept in their products. Cisco routers have multiple LAN ports, one of which is designated as a DMZ port, and the IOS operating system uses Port Address Translation (PAT) to allow traffic to be routed to multiple servers with a single IP address destination. As the name implies, it uses port numbers (such as 80 for the Web server and 25 for the mail server) to distinguish between the multiple servers. This allows you to have multiple public servers without paying for multiple public IP addresses.

Many firewall appliances, such as the SonicWall, come with three Ethernet ports: a LAN port (to connect to the internal network), a WAN port (to connect to the Internet) and a DMZ port (to connect to the network housing your public servers).

Microsoft's ISA Server 2004's multi-networking feature allows you to connect the ISA Server firewall to as many networks as you wish, limited only by the number of network interface cards you can install in the machine. No network is automatically "trusted" in the new ISA model, so you configure security according to the needs of the particular network.

Common DMZ Security Architectures
A DMZ is considered by many to be a "wide open" network, much like the geopolitical DMZ where you risk being shot anytime you set foot inside it. However, all DMZs are not created equal when it comes to the security architecture. Even when you place computers in the DMZ, there are still ways to protect them. The level of security within the DMZ also depends on the nature of the servers that are placed there. We can divide DMZs into two security categories:

DMZs designed for unauthenticated or anonymous access
DMZs designed for authenticated access
If you have a Web server that you want everybody on the Internet to be able to access, (such as a Web presence advertising your company), you'll have to allow anonymous access. You can't easily provide authentication credentials to every stranger who happens upon your site. However, if your Internet-facing servers on the DMZ are used by partners, customers, or employees working off-site, you can require authentication to access them. This makes it more difficult for a hacker to gain access.

The DMZ Honeynet
There is a special use for the anonymous DMZ that's being more popular: creating a "honeynet." This is a network that consists of one or more "honeypot" computers that are designed to lure hackers --either so they can be caught or tracked, or to divert them from the network's real resources. Unlike with other DMZs, you actually want this network to be compromised.

Often the computers on the honeynet are virtual machines that are all installed on a single physical machine, and intrusion detection systems and other monitoring systems are put in place to gather information about the hackers' techniques, tactics and identities.

Host Security on the DMZ
Because the DMZ is a less secure network than the internal network, host security is even more important for the computers that are "out there." The servers on your DMZ should be hardened as much as possible (while maintaining their accessibility to those who need to access them). This means:

All unnecessary services should be disabled.
Necessary services should be run with the lowest privileges possible.
Strong passwords or passphrases should be used.
Unnecessary user accounts should be deleted or disabled and default accounts should be disguised by renaming, changing the description, etc.
Systems should have the latest security updates and patches applied.
Security logging should be enabled (and you should check the logs frequently!)
The Evolution of the DMZ
The definition of "DMZ" is becoming broader, as more uses are found for these "semi-trusted" networks. Today's networks are complex, and security specialists are beginning to realize that the concept of the network "edge" or "perimeter" is outdated; an enterprise network has multiple perimeters. Thus, DMZs may be appropriate at places other than at the edge of the Internet, and large networks can benefit from having multiple DMZs.

Active directory tutorial

2009-11-04T06:51:00.000-08:00

http://www.learnthat.com/Software/learn/1295/Introduction-to-Active-Directory/page/1/

SSL and its generation/installation

2009-10-19T19:52:00.000-07:00

Zurück
How to create a self-signed SSL Certificate ...

... which can be used for testing purposes or internal usage

Overview

The following is an extremely simplified view of how SSL is implemented and what part the certificate plays in the entire process.

Normal web traffic is sent unencrypted over the Internet. That is, anyone with access to the right tools can snoop all of that traffic. Obviously, this can lead to problems, especially where security and privacy is necessary, such as in credit card data and bank transactions. The Secure Socket Layer is used to encrypt the data stream between the web server and the web client (the browser).

SSL makes use of what is known as asymmetric cryptography, commonly referred to as public key cryptography (PKI). With public key cryptography, two keys are created, one public, one private. Anything encrypted with either key can only be decrypted with its corresponding key. Thus if a message or data stream were encrypted with the server's private key, it can be decrypted only using its corresponding public key, ensuring that the data only could have come from the server.

If SSL utilizes public key cryptography to encrypt the data stream traveling over the Internet, why is a certificate necessary? The technical answer to that question is that a certificate is not really necessary - the data is secure and cannot easily be decrypted by a third party. However, certificates do serve a crucial role in the communication process. The certificate, signed by a trusted Certificate Authority (CA), ensures that the certificate holder is really who he claims to be. Without a trusted signed certificate, your data may be encrypted, however, the party you are communicating with may not be whom you think. Without certificates, impersonation attacks would be much more common.

Step 1: Generate a Private Key

The openssl toolkit is used to generate an RSA Private Key and CSR (Certificate Signing Request). It can also be used to generate self-signed certificates which can be used for testing purposes or internal usage.

The first step is to create your RSA Private Key. This key is a 1024 bit RSA key which is encrypted using Triple-DES and stored in a PEM format so that it is readable as ASCII text.

openssl genrsa -des3 -out server.key 1024

Generating RSA private key, 1024 bit long modulus
.........................................................++++++
........++++++
e is 65537 (0x10001)
Enter PEM pass phrase:
Verifying password - Enter PEM pass phrase:

Step 2: Generate a CSR (Certificate Signing Request)

Once the private key is generated a Certificate Signing Request can be generated. The CSR is then used in one of two ways. Ideally, the CSR will be sent to a Certificate Authority, such as Thawte or Verisign who will verify the identity of the requestor and issue a signed certificate. The second option is to self-sign the CSR, which will be demonstrated in the next section.

During the generation of the CSR, you will be prompted for several pieces of information. These are the X.509 attributes of the certificate. One of the prompts will be for "Common Name (e.g., YOUR name)". It is important that this field be filled in with the fully qualified domain name of the server to be protected by SSL. If the website to be protected will be https://public.akadia.com, then enter public.akadia.com at this prompt. The command to generate the CSR is as follows:

openssl req -new -key server.key -out server.csr

Country Name (2 letter code) [GB]:CH
State or Province Name (full name) [Berkshire]:Bern
Locality Name (eg, city) [Newbury]:Oberdiessbach
Organization Name (eg, company) [My Company Ltd]:Akadia AG
Organizational Unit Name (eg, section) []:Information Technology
Common Name (eg, your name or your server's hostname) []:public.akadia.com
Email Address []:martin dot zahn at akadia dot ch
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:

Step 3: Remove Passphrase from Key

One unfortunate side-effect of the pass-phrased private key is that Apache will ask for the pass-phrase each time the web server is started. Obviously this is not necessarily convenient as someone will not always be around to type in the pass-phrase, such as after a reboot or crash. mod_ssl includes the ability to use an external program in place of the built-in pass-phrase dialog, however, this is not necessarily the most secure option either. It is possible to remove the Triple-DES encryption from the key, thereby no longer needing to type in a pass-phrase. If the private key is no longer encrypted, it is critical that this file only be readable by the root user! If your system is ever compromised and a third party obtains your unencrypted private key, the corresponding certificate will need to be revoked. With that being said, use the following command to remove the pass-phrase from the key:

cp server.key server.key.org
openssl rsa -in server.key.org -out server.key

The newly created server.key file has no more passphrase in it.

-rw-r--r-- 1 root root 745 Jun 29 12:19 server.csr
-rw-r--r-- 1 root root 891 Jun 29 13:22 server.key
-rw-r--r-- 1 root root 963 Jun 29 13:22 server.key.org

Step 4: Generating a Self-Signed Certificate

At this point you will need to generate a self-signed certificate because you either don't plan on having your certificate signed by a CA, or you wish to test your new SSL implementation while the CA is signing your certificate. This temporary certificate will generate an error in the client browser to the effect that the signing certificate authority is unknown and not trusted.

To generate a temporary certificate which is good for 365 days, issue the following command:

openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt
Signature ok
subject=/C=CH/ST=Bern/L=Oberdiessbach/O=Akadia AG/OU=Information
Technology/CN=public.akadia.com/Email=martin dot zahn at akadia dot ch
Getting Private key

Step 5: Installing the Private Key and Certificate

When Apache with mod_ssl is installed, it creates several directories in the Apache config directory. The location of this directory will differ depending on how Apache was compiled.

cp server.crt /usr/local/apache/conf/ssl.crt
cp server.key /usr/local/apache/conf/ssl.key

Step 6: Configuring SSL Enabled Virtual Hosts

SSLEngine on
SSLCertificateFile /usr/local/apache/conf/ssl.crt/server.crt
SSLCertificateKeyFile /usr/local/apache/conf/ssl.key/server.key
SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown
CustomLog logs/ssl_request_log \
"%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"

Step 7: Restart Apache and Test

/etc/init.d/httpd stop
/etc/init.d/httpd stop

https://public.akadia.com
http://tldp.org/HOWTO/SSL-Certificates-HOWTO/

grub and bcd edit

2009-10-14T16:43:00.000-07:00

Linux
Added by Mahmoud Al-Qudsi, last edited by Mahmoud Al-Qudsi on Jan 25, 2009 (view change)
Labels

(None)
Add Labels

Wait Image
Enter labels to add to this page:
Looking for a label? Just start typing.

Adding Linux to the Vista Bootloader

One of the most features that EasyBCD brings to the table is full-blown support for Linux, BSD, Solaris, and more starting from version 1.5, putting it a league above the competition. Most importantly, EasyBCD offers several different ways to get these operating systems working with Vista, just in case one or more fail to work out the way you want them to.

Chainloading is a dual-boot term that refers to one bootloader handing off the boot process to another. In this case, we configure the Vista bootloader to ask either Grub or Lilo (the most common Linux bootloaders) to complete the boot process for us - minimizing configuration requirements and ensuring maximum compatibility.
Using a Specific Distro?

We have distro-specific guides for Fedora and Ubuntu available!
Vista before Linux

EasyBCD makes installing Linux after you have Windows Vista up-and-running a breeze. These steps assume you have Windows Vista properly installed and booting, and are looking to install Linux on a second hard drive or partition. These steps also assume that you are using the default Windows Vista bootloader, and don't manually change the active partition around. If you had Linux installed before you installed Windows Vista, scroll down to the next section.

1. Put the Linux CD in the drive, and start the installation normally.
2. When prompted to set up the bootloader, make sure you specify to install LILO, GRUB, or whatever to the bootsector of the partition that Linux is being installed to and not the MBR of your hard drive.
3. Finish the Linux installation, take the CD out of the drive, and reboot.

At this point, you'll go straight back to Windows Vista. Don't panic, everything is OK - you'll be in Linux soon enough!

1. Turn on EasyBCD, go to the "Add/Remove Entries" screen and pick Linux from the tabs at the top.
2. Pick the appropriate bootloader from the drop-down menu (either GRUB or LILO),
3. Give the entry a user-friendly name (and if you want to keep "NST Linux Loader" as the text, we won't say no!)
4. The hardest part of this mind-numbingly difficult exercise (/sarcasm) is choosing the correct hard drive and partition numbers that correspond to the partition you installed Linux (and most importantly, the bootloader) to.
In EasyBCD (and Windows in general), drive numbers start at 0, and partitions start at 1. So the second partition of the first drive would be 0, 2.
5. Press "Add Entry" and reboot.

When the Vista bootloader asks you what OS you'd like to boot into, select Linux to continue the first-run configuration for your brand-spanking-new Linux install.

It's that simple!
Linux before Vista

These steps assume you had a fully working Linux install before you installed Windows Vista. It also assumes that you've been using either GRUB or LILO to boot Linux in the past. If you used some other bootloader, please consult the appropriate documentation on their respective website(s).

At this point, you have two options. The first option is more compatible & more reliable, but it can be a hassle depending on just how much Linux experience you have. The second uses the all-new NeoGrub bootloader in conjunction with the Vista BCD to get Linux running - but it only works on a limited subset of setups. Skip down to the NeoGrub section for more info.
Reinstall the Bootloader
Useful Information
We don't want to install the Linux bootloader back onto the MBR, because that'll just erase the Vista bootloader - leaving you with Linux and nothing else. So we install the bootloader onto a hidden sector called a "bootsector" and we tell Windows Vista where it's installed - then Windows Vista launches (chainloads) this sector on the disk, starting the Linux bootloader up for us.

At this point, we need to install GRUB to the bootsector of our system. You have two ways to proceed: the first is to boot into Linux via a Live CD or a recovery disk. The second (easier) method is to use Super Grub Disk to reinstall GRUB.
Booting into Linux

Reinstalling GRUB in a correct requires that you run the GRUB program from within Linux. If you can boot into your Linux installation or you have a working Live CD, you may do so now. Once the desktop loads, open a console window (like a dos prompt in Windows).

Once the console is open, run the following command:

su -
bash
grub

You should now be in a GRUB shell - basically an OS within an OS. If you're using Ubuntu or other distributions that work in a similar fashion, use this instead:

sudo bash
grub

At this point, skip this next section and move on to "Installing GRUB"
Booting into Super Grub Disk

If you don't have a Live CD or you're not comfortable with Linux - no problem.

"Super Grub Disk" is a really nifty utility for recovering your GRUB bootloader from a bootable floppy, CD, DVD, or even USB Stick. NeoSmart Technologies has provided a mirror of Super Grub Disk for your convenience - it's only 400kb. Extract the archive to your desktop, and burn the ISO to a blank CD with your favorite application. If you so choose, you can elect to install it to a floppy or USB stick instead by following these instructions.

Once you have SGD installed to a CD, reboot your PC and boot from the CD. Choose "English" from the Language selection screen, then once SGD has loaded, press a key to continue. At the SGD menu, press the letter 'c' to enter a GRUB prompt.

At this point, a GRUB shell will open, and you can proceed to the next step.
Installing GRUB

Now that you're within the GRUB program (by means of a Live CD or SGD), continue with the following:

find /boot/grub/stage1

It'll return a (hdx,y) value you can use to setup GRUB. (If it doesn't, you may need to play around with the path to stage1 which changes from distro to distro, but most likely this is what you're looking for.)

In Linux, both drive and partition numbers start from zero. For example, if you had Windows on the first partition of the first drive, and Linux on the second partition of the same drive, you would use hd(0,1) to refer to the second partition of the first drive. Read Drive Letters and Numbers for more info and a colorful history of this topic.

(hd0,0) first primary partition on first hard disk
(hd0,1) second primary partition
(hd0,2) third primary partition
(hd0,3) fourth primary partition (usually an extended partition)
(hd0,4) first logical partition
(hd0,5) second logical partition ...

Once you've got that value, type this in, substituting (hdx,y) for the output of the previous command.

root (hdx,y)
setup (hdx,y)
setup (hdx,y)
quit

The setup line is repeated twice because often times the first GRUB setup command will error out, and the second will succeed - because of changes made by the first. No harm done running it twice. We've just installed GRUB to the bootsector of the partition. (If we wanted to install it to the MBR, we would have written hdx only - but we don't want to do that!)

If you're using a Live CD, restart your PC now. The shutdown -r 0 command will reboot your PC. Make sure you've removed the CD from the drive so that you can get back into Windows Vista.
Back in Windows Vista

At this point, you should be back in Windows Vista with either GRUB or LILO tucked away in a hidden sector somewhere on your drive. Fire up everyone's favorite bootloader manager (why, EasyBCD, of course!) and go to the "Add\Remove Entries" screen.

1. Choose "Linux\BSD" from the tabs on the top.
2. Select either GRUB or LILO from the drop-down menu depending on what you configured earlier.
3. Choose the correct drive and partition number for your Linux partition. This should be the same partition as the one we just finished configuring. Refer to the image above for an example.
4. Enter the name you would like to give the Linux entry in the bootloader.
5. Press "Add Entry" and reboot to test.

These steps can be a bit tricky, and making mistakes could lead to some unwanted trouble (but don't worry, nothing that can't be fixed without dataloss). If you're hesitant about something, don't risk it - just ask for help.
Method 2: Using NeoGrub to boot Linux

NeoGrub is NeoSmart Technologies' implementation of the open-source GRUB bootloader (ported over to Windows by the Grub4Dos team) intended to allow Windows users to boot into Linux without having to resort to rescue discs, second bootloaders, or messy install routines for GRUB or LILO.

NeoGrub is the recommended solution for *nix-phobics - and for people that don't have a rescue/live CD on hand. It's a nifty way of getting Linux to boot quickly and efficiently without spending hours switiching CDs and rebooting your PC over and over again.

NeoGrub can be used to read existing menu.lst files from old dual-boot setups and allows for fine-grain control over dual-boot settings for experienced users.
External Links

These are links to external (non-NST) guides that revolve around dual-booting Windows Vista and Linux - with the help of EasyBCD. Please note that these articles were written with regards to EasyBCD 1.5x — EasyBCD 1.6 has some new features that make certain steps used in these guys no longer necessary.

* Herman's excellent Guide to GRUB (via SGD)
* APC Magazine's Vista & Linux Dual Boot Guides
* GNU GRUB Native Installation Docs (no details, rather useless)

Children (2)
Hide Children | View in Hierarchy
[Fedora] Fedora
[Ubuntu] Ubuntu

Htaccess

2009-09-22T13:39:00.000-07:00

.htaccess is used to allow access to httpd apart from hosts.allow and hosts.deny :)

Heap

2009-08-31T21:01:00.001-07:00

http://www.openasthra.com/multithreading/heap-overview/

An Overview of Heap Overflow

Each threads of a running program has a stack where local variables are stored. In
virtual memory space
linux all program has a .BSS (global and static uninitialized varibles) and
.DATA segment (global and static initialized varibles) along with other segments
used by malloc() and allocated with brk() - Sets the end of data segment to the
values specified or mmap() - map file into memory, it ask length bytes
starting at offset from the file specified by the file descriptor fd into
memory, preferably at address start.

malloc() breaks up a chunk of memory allocated using brk() into small parts and
gives the user one of those part when a request is made.
It used various
techniques like smallest-first, best-fit, next best fit, first-fit etc.
Lot
of meta-data about the location of the chunks, ths size of the chunks and other
special area for small parts.
All these is information are organized into
buckets and in some others into balanced tree structure.

So like stack
overflow we can overflow heap also, by overwriting those meta-data.

One such vulenrable program is cprog1.c

[highlight lang=”CPP”]
#define BLOCKSIZE 666
int main(int argc, char *argv[])
{
char *pcBuf1, *pcBuf2;
pcBuf1 = (char *) malloc (BLOCKSIZE);
pcBuf2 = (char *) malloc (BLOCKSIZE);
printf(”Buffer 1: %p Buffer 2: %pn”,pcBuf1,pcBuf2)
strcpy(pcBuf1,argv[1]);
free(pcBuf2);
free(pcBuf1);
}
[/highlight]

$> gcc -o cprog1 cprog1.c
$> ./cprog1
Buffer1: 0×9017008 Buffer2: 0×90172a8***glibc detected *** double free or corruption (out): 0×90172a8 ***Aborted

Here we are overflowing

Buffer2 with the data in

argv[1]

.
malloc implementations, including Linux’s dlmalloc, store
extra information in a free chunk. As free chunks can be used to store
information about other chunks.

Note: The output of all the programs
discussed here will be complier/platform dependent.

Instead of malloc we
could have used something like

p = (char *) sbrk(BLOCKSIZE);

But it isn’t efficient,
portable. What about free?

Allocation Algorithms :

Following are some of the memory allocation
algorithms used.

* First Fit
o Keep a linked list of free node
o Search for the first one that’s BIG enough
* Best Fit
o Keep a linked list of free node
o Search for the smallest one that’s BIG enough
* Free list: A circular list

[tags] heap usage, heap, heap overview, heap in

Some algorithms

2009-08-31T20:59:00.001-07:00

. Reverse a singly linked list

//
// iterative version
//
Node* ReverseList( Node ** List )
{
Node *temp1 = *List;
Node * temp2 = NULL;
Node * temp3 = NULL;

while ( temp1 )
{
//set the head to last node
*List = temp1;

// save the next ptr in temp2
temp2= temp1->pNext;

// change next to privous
temp1->pNext = temp3;
temp3 = temp1;
temp1 = temp2;
}

return *List;
}

2. Delete a node in double linked list

void deleteNode(node *n)
{
node *np = n->prev;
node *nn = n->next;
np->next = n->next;
nn->prev = n->prev;
delete n;
}

3. Sort a linked list

//sorting in descending order
struct node
{
int value;
node* NEXT;
}
//Assume HEAD pointer denotes the first
//element in the linked list
//only change the values? don't have to
//change the pointers
Sort( Node *Head)
{
node* first,second,temp;
first= Head;
while(first!=null)
{
second=first->NEXT;
while(second!=null)
{
if(first->value < second->value)
{
temp = new node();
temp->value=first->value;
first->value=second->value;
second->value=temp->value;
delete temp;
}
second=second->NEXT;
}
first=first->NEXT;
}
}

4. Reverse a string

void ReverseString (char *String)
{
char *Begin = String;
char *End = String + strlen(String) - 1;
char TempChar;

while (Begin < End)
{
TempChar = *Begin;
*Begin = *End;
*End = TempChar;
Begin++;
End- -;
}
}

5. Insert a node a sorted linked list

void sortedInsert(Node * head, Node* newNode)
{
Node *current = head;

// traverse the list until you find
//item bigger the new node value
//
while ((current!= NULL) &&
(current->data < newNode->data))
{
current = current->next;
}
//
// insert the new node before the big item
//
newNode->next = current->next;
current = newNode;
}

6. Covert a string to upper case

void ToUpper(char * S)
{
while (*S!=0)
{
*S=(*S >= 'a' && *S <= 'z')?(*S-'a'+'A'):*S;
S++;
}
}

7. Multiple a number by 7 without using * and + operator.

// mulitplied by 2 ^ 3 = 8
NewNum = Num << 3;

// 8 � 1 = 7
NewNum = NewNum - Num;

8. Write a function that takes in a string parameter and checks to see whether or not it is an integer, and if it is then return the integer value.

9574719ba5683186a52a574383d220e9007

9. Print a data from a binary tree - In-order(ascending)

//
// recursive version
//

Void PrintTree ( struct * node node )
{
if ( node == NULL )
return;

PrintTree(node->left );
printf("%d", node->data);
PrintTree(node->right );
}

10. print integer using only putchar

//
// recursive version
//
void PrintNum ( int Num )
{
if ( Num == 0 )
return;
PrintNum ( Num / 10 );
puthcar ( '0'+ Num % 10 );
}

11. Find the factorial of number

//
// recursive version
//

int Factorial( int Num )
{
if ( num > 0 )
return Num * Factorial ( Num �1 );
else
return 1;
}

//
// iterative version
//

int Factorial( int Num )
{
int I
int result = 1;

for ( I= Num; I > 0; I- - )
{
result = result * I;
}

return result;
}

12. Generate Fib numbers:

// recursive version
int fib( n )
{
if ( n < 2 )
return 1;
else
return fib ( n-1 ) + fib ( n-2 );
}

//iterative version
int fib( n )
{
int f1 =1, f2 = 1;

if ( n < 2 )
return 1;
for ( i = 1; i < N; i++)
{
f = f1 + f2;
f1= f2;
f = f1;
}
return f;
}

13. Write a function that finds the last instance of a character in a string

char *lastchar(char *String, char ch)
{
char *pStr = NULL;

// traverse the entire string

while( * String ++ != NULL )
{
if( *String == ch )
pStr = String;
}

return pStr;
}

14. Return Nth the node from the end of the linked list in one pass.

Node * GetNthNode ( Node* Head , int NthNode )
{
Node * pNthNode = NULL;
Node * pTempNode = NULL;
int nCurrentElement = 0;

for ( pTempNode = Head;
pTempNode != NULL;
pTempNode = pTempNode->pNext )
{
nCurrentElement++;
if ( nCurrentElement - NthNode == 0 )
{
pNthNode = Head;
}
else
if ( nCurrentElement - NthNode > 0)
{
pNthNode = pNthNode ->pNext;
}
}
if (pNthNode )
{
return pNthNode;
}
else
return NULL;
}

15. Counting set bits in a number.

//First version:
int CoutSetBits(int Num)
{
for(int count=0; Num; Num >>= 1)
{
if (Num & 1)
count++;
}
return count;
}

//Optimized version:
int CoutSetBits(int Num)
{
for(int count =0; Num; count++)
{
Num &= Num -1;
}
}

auto_ptr in C++

2009-08-31T20:56:00.001-07:00

In C++, local instances (these not allocated by hand) have their destructors run as soon as they go out of scope. This behavior is explored in a few different schemes, besides its obvious intent of destructing the instance. The most known one is the auto_ptr class, part of the STL. It is merely a memory guard which takes care of deallocating the given pointer once its own scope is over. In the short example below, the s variable won’t leak.

#include
using namespace std;
void f()
{
char *s = strdup("foobar");
auto_ptr sp(s);
cout << sp.get();
}

While this is an interesting place to use this concept, it’s a good algorithm to be aware of generically. A good example of how useful this could be is how the apt-shell (part of APT-RPM) cache guard mechanism was implemented. This program was implemented as a refactoring of apt-get, but they have a fundamental difference: while apt-get is a one-shot program (run just once, and get back to the user), apt-shell maintains the internal cache (the structure where information about the packages is kept) alive for a long time, and changes it in many different ways during its execution time. Thus, I had to implement some way to make it easy to protect this cache against being unadvisedly changed in all these functions coming from apt-get. To do that, I have implemented an AutoRestore class which takes care of restoring the original state of the cache, unless explicitly told to release it. Here is the complete class, from apt-shell.cc:

class AutoRestore {

pkgDepCache::State State;

bool Guarded;

public:

inline pkgDepCache::State *operator ->() {return &State;};

inline pkgDepCache::State *operator &() {return &State;};

inline void UnGuard() { Guarded = false; };

AutoRestore(pkgDepCache &Cache) : State(&Cache), Guarded(true) {};

~AutoRestore() { if (Guarded) State.Restore(); };

};

To use it, it’s just a matter of creating a local instance, passing the cache to the constructor. It will take care of undoing unexpected changes. Really comfortable! [tags]auto_ptr[/tags]

inline functions

2009-08-31T20:55:00.000-07:00

Yes and no. Sometimes. Maybe.

There are no simple answers. inline functions might make the code faster, they might make it slower. They might make the executable larger, they might make it smaller. They might cause thrashing, they might prevent thrashing. And they might be, and often are, totally irrelevant to speed.

inline functions might make it faster: procedural integration might remove a bunch of unnecessary instructions, which might make things run faster.

inline functions might make it slower: Too much inlining might cause code bloat, which might cause “thrashing” on demand-paged virtual-memory systems. In other words, if the executable size is too big, the system might spend most of its time going out to disk to fetch the next chunk of code.

inline functions might make it larger: This is the notion of code bloat, as described above. For example, if a system has 100 inline functions each of which expands to 100 bytes of executable code and is called in 100 places, that’s an increase of 1MB. Is that 1MB going to cause problems? Who knows, but it is possible that that last 1MB could cause the system to “thrash,” and that could slow things down.

inline functions might make it smaller: The compiler often generates more code to push/pop registers/parameters than it would by inline-expanding the function’s body. This happens with very small functions, and it also happens with large functions when the optimizer is able to remove a lot of redundant code through procedural integration ? that is, when the optimizer is able to make the large function small.

inline functions might cause thrashing: Inlining might increase the size of the binary executable, and that might cause thrashing.

inline functions might prevent thrashing: The working set size (number of pages that need to be in memory at once) might go down even if the executable size goes up. When f() calls g(), the code is often on two distinct pages; when the compiler procedurally integrates the code of g() into f(), the code is often on the same page.

inline functions might increase the number of cache misses: Inlining might cause an inner loop to span across multiple lines of the memory cache, and that might cause thrashing of the memory-cache.

inline functions might decrease the number of cache misses: Inlining usually improves locality of reference within the binary code, which might decrease the number of cache lines needed to store the code of an inner loop. This ultimately could cause a CPU-bound application to run faster.

inline functions might be irrelevant to speed: Most systems are not CPU-bound. Most systems are I/O-bound, database-bound or network-bound, meaning the bottleneck in the system’s overall performance is the file system, the database or the network. Unless your “CPU meter” is pegged at 100%, inline functions probably won’t make your system faster. (Even in CPU-bound systems, inline will help only when used within the bottleneck itself, and the bottleneck is typically in only a small percentage of the code.)

There are no simple answers: You have to play with it to see what is best. Do not settle for simplistic answers like, “Never use inline functions” or “Always use inline functions” or “Use inline functions if and only if the function is less than N lines of code.” These one-size-fits-all rules may be easy to write down, but they will produce sub-optimal results.

Difference between C in UNIX and C on DOS

2009-08-31T20:30:00.001-07:00

There is no difference but UNIX is a multitasking, multi-user OS unlike DOS. In DOS, serial multi-tasking can be achieved but it makes the computer work less efficiently. If, for example, there was some calculation being done and we called in Sidekick, then all work on the calculations would stop as the computer responded to Sidekick. Wouldn’t it then be far better to give Sidekick only a part of the computer’s time? So that even while we were in Sidekick the calculations would carry on being performed in the background.

Under UNIX, all we have to do is to compile and link the background while we carry on doing whatever else we want in the foreground.

Background Process:

main()
{
long l;
for(i=0;i<=4000000;i++);
printf("l is %d \n",i);
}

1)Compile it at UNIX prompt by saying CC.

2)The file generated will be a.out.

3)Execute it through the command a.out &. The & will make it a background process which is terminated only if process ends, or is interrupted with a DEL.

The UNIX prompt will come back on screen immediately. Load the editor again and wait. After some time a l is 4000000 will be displayed at the top left corner of the screen.

Isn’t this more convenient than DOS’s single tasking environment?

UNIX is able to switch between processes because each process has an identification number. This identification number identifies the variables and REGISTER values that are associated with this process. These variables are in a state of limbo when the process is suspended. And get activated again once the microprocessor resumes execution of the process.

Process Identification:

/*There is a UNIX function getpid()- that enables us to get the identification number of a process*/

main()
{
int pid;
pid=getpid();
print("Process ID is %d\n",pid);
}

The pid value will depend on the number of processes already running. It will always be unique.

Parent And Child:

A process in UNIX is not a stand-alone. This results in a parent-child relationship existing between processes.

main()
{
int ppid;
ppid=getppid();
print("Parent Process ID is %d\n",ppid);
}

This program will give the ID number of its parent. So who is this parent?

When we boot the system, a special process called the SWAPPER or SCHEDULER is created with a PID of 0. The SWAPPER manages memory allocation for processes and influences CPU allocation. The SWAPPER in turn creates three children:the process dispatcher, vhand and bdflush with ID numbers 1,2,3 respectively.

This is done by executing the file init which exists in the etc sub-directory. The dispatcher now gives birth to the shell. From now on all processes initiated by us are children of the shell.

The fork():

Processes initiated by us can also create children in the same manner as the swapper and the process dispatcher did. These children processes are created using the fork() function. It is by forking processes that we can exploit the multitasking capability of UNIX.

main()
{
fork();
printf("Hello World\n");
}

Can you guess what the output of this program will be?

The statement Hello World will be displayed twice on screen. The fork() creates a child that is a duplicate of the parent process. The parent process in this case being the program listed above. Since now there are two identical processes in memory, the Hello World is printed twice.

The moment a call is made to the fork(), a child process is created. The child copy, too, gets a copy of the variable pid, but with a value of 0 by default. The parent process, is returned a value greater than 0 by the fork().

For UNIX is more than just a multi-tasking OS. It also has the facility to enable many users to access the machine it is housed in at one time. And not just access the machine at same time but also interact with each other.

But how UNIX is able to take care of different users simultaneously?

The kernel creates various processes that ultimately allows us to login. The first one is INIT process. INIT decides which of the terminals that can be used to login. The program GETTY, tells UNIX which program to execute. GETTY in turn reads data from the etc/gettydefs file, displays the login: prompt on screen and waits for input. After which it spawns the login program, which takes the login name we have keyed in as a parameter. Then INIT processes creates a child process by forking, through which login processes is executed with the help of exec() fuction.

Bad APIs at a glance

2009-08-31T20:27:00.000-07:00

Bad APIs at a glance

strcpy — Too easy to create a buffer overrun
strcat – Too easy to create a buffer overrun
strncpy – Deceptive. Doesn’t always NUL terminate!
strncat – Deceptive. Doesn’t always NUL terminate!
wstrcpy – Too easy to create a buffer overrun
wstrcat – Too easy to create a buffer overrun
wstrncpy – Deceptive. Doesn’t always NUL terminate!
wstrncat – Deceptive. Doesn’t always NUL terminate!
sprintf – Difficult to avoid buffer overruns with complex format strings
vsprintf – Difficult to avoid buffer overruns with complex format strings
wsprintf – Difficult to avoid buffer overruns with complex format strings
gets – Impossible to be safe with gets
strtok – Not reentrant

strcpy and friends

The problem

There is no way to limit the number of characters copied to the
destination buffer. Therefore there is potential for a buffer overflow
when the source has a length greater than that of destination buffer.

In the first code example below, the stack buffer aDest can be
overrun if GetString returns a string longer than 9 characters. If an
attacker can manipulate what GetString returns, perhaps because it is
read out of a network protocol or out of a multimedia file, they can
then manipulate the call stack. In the worst case they can cause a jump
to any address on the device by overwriting the return address that is
stored on the stack.

Strncpy, strncat

strncpy is deceptive

Strncpy will never copy past the end of the buffer given it which is
good. However if the input won’t fit into the buffer it will fill the
buffer and NOT terminate it with a NUL. Your string won’t be what you
think it is and code reading the string will read off the end of the
buffer. std_strlcpy will always NUL terminate whether or not the
destination buffer is large enough.

These comments apply to strncat, wstrncpy and wstrncat too.

gets

Gets has no way to bound how much is writes and worse what it writes
by definition comes from an external source. This is probably the most
dangerous function of the lot. Fortunately it is not widely used.

sprintf

Why sprintf is bad

There is no way to limit the number of characters written to the
destination. Therefore there is potential of buffer overflow when the
resulting output has a length greater than that of destination buffer.

strtok

Why strtok is bad

This function maintains internal state as a global variable
. Because
of that it is not re entrant and not safe in a multi-threaded
environment. Calls to it from one thread to parse one string may
interleave with calls to it from another thread to parse another
string. The result will be that pointers one caller will get a pointer
to the other callers string.

Contemporary code is multi-threaded and thus it is not safe to use this. A bug
caused by the use of this function will be very difficult to track down
because it could be in code completely unrelated to the code
manifesting the problem.

What to do instead

The function strtok_r is safe because the caller passed storage for
the single pointer it can use to maintain the state it needs. Note that
strtok_r should always be checked for a NULL return value.

Scanf

This discussion applies to sscanf as well as scanf.
Why scanf is bad

If you use %s with scanf without a field length qualifier there is
no limit on how much data it will write to the buffer for the token
matching %s.

Also, as with any form of printf, the format string should be
constant, or if not constant, unmodifiable from out side the software
or the device. If the attacker can change the format string they can
trivially overrun a buffer.
Using scanf safely

Always specify a length for %s.

Always make the format string a compile time static const string (just use strings in quote marks like "%d %d").

In general scanf is cumbersome and error prone for parsing input.
Often other parsing approaches are better. If all you need is string to
integer conversion, strtoul is better.

[tags]bad standard apis, bad apis, dangerous standard APIs[/tags]

bffer overrun

2009-08-31T20:26:00.001-07:00

What is a buffer overrun

Simply stated it is running a read or write pointer off the end of the allocated buffer into some other buffer. Usually it occurs because pointer arithmatic is bad worse there’s not buffer length checking at all.

Why are they bad

Hackers have figured out many ways to exploit buffer overruns to take over the device. In particular they can be used to load and run hostile code. Since handsets at this point in time still run mostly with a single memory space (little memory protection) a clever attacker can potentiall do the following:

* brick the phone so it has to go back to a service center
* break SIM locking
* take protected content
* reboot the phone

Are some worse than others?

In one sense all buffer overruns are the same and horribly bad because of the single memory space. They all give access to the whole handset.

In another sense they are not the same because some are easier to discover than others. For example well known overruns in open source are the easiest.

In another sense they are not all the same because some are easier to exploit than others. A one byte read overrun is not useful for an attacker. An overrun that smashes the stack that is easy to exploit via SMS is very useful.

buffer overruns

2009-08-31T20:22:00.000-07:00

http://www.openasthra.com/cc-programming-concepts/buffer-overruns-part1/

Buffer Overruns/Overflows are typically caused by trusting input data to a function that is external and is unchecked. Most of the times, this is unintentionally invoked by bad sloppy code. However, when done intentionally by a hacker, this can cause havoc. Some of the most common, buffer overrun prone functions include strcpy, memcpy, strcat etc. (bad standard APIs are explained in another post) In unintentional buffer overruns/overflows, this mostly results in writing to memory not owned by your processes address space. In such cases this would end, in an access violation or a core dump, causing the program to be aborted in most cases. However, these buffer overruns can be exploited to run arbitrary code on the machine, even code that is injected in by the attacker. Before going deep into this overruns let us understand the program executable layout, this helps us in better understanding of the overurns.

Memory layout for a executable on linux is described below. The layout is called ELF (Executable and linkable format), windows counterpart format would be little different but it is similar to the ELF.

program layout

Text segment is read-only segment and it includes all the program instructions. All initialized variables (but not local variables) are stored in initialized data segment and all uninitialized variables are stored in BSS (block start by symbol). All malloc/new variables in heap segment, all auto/local variables will be in Stack. Lets take an example and see how variables are stored.

Static int a;

Int x = 1;

Int y;

Int main()

{

Int b;

Char c[10];

Int *d = (int *)malloc(sizeof(int));

/*?. ?.*/

}

In the above program x is stored in initialized data segment, y is stored in BSS, b & c are stored in stack, d is stored in heap. The stack frame in C contains args, return address, base pointer. Feeding more data to stack vars than they can hold will cause overwrite return address.

Stack Overruns: These type of buffer overruns occur, when a buffer that is declared on the stack is overwritten by copying more data than the buffer can hold. It so happens, that the variables (buffer in this case) declared on the stack are located next to the return address of the function?s caller (this is what we have discussed above). As I mentioned above, this usually occurs when user input is unchecked, e.g. strcat. And because the return address is next to the buffer on the stack, overwriting the buffer, means overwriting the return address, which is what gets executed with the function returns. An attacker can carefully exploit the overrun in such a way the data that is overwriting the return address, is an address of a function that he wants to execute.

Heap Overruns: Like Stack Overruns, Heap overruns can also cause memory and stack corruption. But unlike contrary developer belief, although heap overruns are harder to exploit, they are definitely exploitable.
Array Indexing Errors (Overruns and Underruns): These type of errors are less exploited compared to static buffer overruns. You can think of an array as a block on memory (buffer) that you can index into and then read/write from that location. Bad array implementation do not check indices well before access the memory locations. Sloppy code like this can be exploited to run arbitrary code and create havoc. (Well, in some cases you may never ever now that your machine is freely accessible and controlled by the attacker ). Again, unlike contrary developer belief, don?t be fooled that only memory past the end of the array can be exploited.
Format Strings Exploits: These type of exploits are not exactly buffer overruns, but they lead to the same class of problems. These errors are usually caused in functions that take in variable number of arguments, because there isn?t a good way for the function to determine the number of arguments passed into these functions. (printf family in the CRT). The %n type field for printf represents the number of characters successfully written so far to the stream or buffer. This value is stored in the integer whose address is given as the argument. So, this can basically be used to write into memory of the processes address space. But how can an attacker inject such code, the answer lies to the way the sloppy programmer writes code ala printf(input), rather than printf(?%s?, input). The latter prevent the user from using his own format, since its already defined, unlike the earlier case, where the input can be manipulated to create the format string.

Here are some buffer overrun examples explained?.

Casting exploitation:

fig1

Overflow:

fig2

Please post your comments so that we can eloborate more on required areas.

padding

2009-08-31T20:21:00.001-07:00

What is Padding?
Aligning members of a structre to a byte boundary. Padding is done by compiler to gain the performance. Most hardware architectures
access addresses fast if they are aligned properly, otherwise there will be performance penalty. In order to increase performance compiler pads structures.

How Padding done?
Normally processors require members of the structure aligned to byte boundary, for instance char member(1 byte) should be aligned to
one byte boundary that is it can appear at any byte boundary, a short int(2 bytes) should be aligned to 2 byte boundary that is it should appear at 2 byte boundary, long int(4 bytes) should be aligned to 4 byte boundary. If we declare a basic data type in structure it should appear at sizeof(basic data type) byte boundary, in order to make the start address of the next member at an aligned address.

Example:
Struct X
{
Char a;
Short int b;
Char c;
Long
d;
};

| a(1) | P(1)| b(2) | c(1)| P(3) | d(4) |

P - padding bytes

Why Padding?
If the processor is serious about alignment it raises an exception/signal (ex. SIGBUS), otherwise there will be a performance penalty
will be there as misalignment slowdown data access.

Are members order going to be changed?
ANSI C requires fields of a structure need to allocated in the order they are declared.

As a programmer how does padding matter to me?
It matters if you are concerned about space it takes. By changing the members order of declarations, some amount space can be saved. For example
Struct X
{
Char a;
Long
Int b;
Char c;
Long int d;
Char e;
Long int f;

};

Sizeof(struct X) = 24 bytes

Lets reorder the above structure as below

Struct X
{
Char a;
Char c;
Char e;
Long Int b;
Long int
d;
Long int f;
};

Sizeof(struct X) = 16 bytes

Padding is only done to members or to the structure as a whole
also?
Padding will be done by compiler to structure’s members and to the structure as a whole also. Compiler pads structure as whole because this allows each member of structure aligned in array of structures.

Packed structures
Some compilers provide #pragma to suppress padding or to make it packed to n number of bytes. Some provide keywords to do this.
Generally pragma which is used for modifying structure padding will be in the below format (depends on compiler)
#pragma pack(n)
For example arm
provides __packed keyword to suppress structure padding. Go through your compiler manual to know more about this.

So a packed structure is a structure without padding.
Generally packed structures will be used
->to save space, and
->to format a data structure to transmit over network using some protocol (this is not a good practice of course because you need to deal
with endianness)

How to declare array of packed structures?
We can not declare array of packed structures as for declaring as an array we need each member of structure aligned. So alternatively either we can declare something like this
Struct X
{
Unsigned char a;
Unsigned long b;
}array[NUM_ELEMENTS];

The above structure yields compilation error, if we declare this as below, we should be able to handle..
Unsigned char array_bytes[sizeof(structure) * NUM_ELEMENTS];
And take special care while handling array_bytes.

When I need to be careful about structure padding?
1)when writing structures to a file on one machine and reading them on different machine
This involves problem, consider this example
Struct X
{
Char a;
Long int b;
};

x.a = 0�01;
x.b = 0�02;

If you write struct X to a file, it would have written 0�01 G G G 0�00 0�00 0�00 0�02 (G is garbled value, because this area is padded)
While reading this on different machine(could be big endian) we read the structure as whole, on big-endian these values will be altered and as a result we read corrupted value for member a;
Solution for this would be using a packed structures we can eliminate this.

How can I pack structures Using GCC?
Using the "packed" attribute against the members of a structure. This attribute mechanism is an extension to the GNU C compiler. An example of how you would do this is below.

struct X
{
unsigned char a __attribute__((__packed__));
unsigned
short b __attribute__((__packed__));
unsigned long c
__attribute__((__packed__));
};

Another way to achieve the same is as shown below:
struct X
{
unsigned char a;
unsigned short b;
unsigned long c;
}
__attribute__((__packed__));

How can I pack structures Using ARM compiler?
__packed keyword is used for this purpose, usage is shown below

__packed struct X
{
char a;
Long int b;
};

May be in some other post, I try to give details about how padding done for
global variables, pointers and etc. on some selected machines. Your comments,
suggestions, criticism, etc. most welcome.

endianness

2009-08-31T20:17:00.001-07:00

//returns zero for little endian
//returns 1 for big endian
//otherwise some error occured
int find_endianness()
{
int x = 0x01 ;
if (* (char *) &x == 1)
{
//little endian
return 0;
}
else if (*((char *)&x+sizeof(int)-1)==1)
{
//big endian
return 1;
}
else
{
//could not find what we have written in that byte
return -1;
}

return 0;
}

Big endian and little endian
"Little Endian" means that the low-order byte of the number is stored in memory at the lowest address, and the high-order byte at the highest address. (The little end comes first.) For example, a 4 byte Long integer

Byte3 Byte2 Byte1 Byte0

will be arranged in memory as follows:

Base Address+0 Byte0

Base Address+1 Byte1

Base Address+2 Byte2

Base Address+3 Byte3

Intel processors (those used in PC’s) use "Little Endian" byte order.

"Big Endian" means that the high-order byte of the number is stored in memory at the lowest address, and the low-order byte at the highest address. (The big end comes first.) Our LongInt, would then be stored as:

Base Address+0 Byte3

Base Address+1 Byte2

Base Address+2 Byte1

Base Address+3 Byte0

Motorola processors (those used in Mac’s) use "Big Endian" byte order.

Which is Better?

You may see a lot of discussion about the relative merits of the two formats, mostly religious arguments based on the relative merits of the PC versus the Mac. Both formats have their advantages and disadvantages.

In "Little Endian" form, assembly language instructions for picking up a 1, 2, 4, or longer byte number proceed in exactly the same way for all formats: first pick up the lowest order byte at offset 0. Also, because of the 1:1 relationship between address offset and byte number (offset 0 is byte 0), multiple precision math routines are correspondingly easy to write.

In "Big Endian" form, by having the high-order byte come first, you can always test whether the number is positive or negative by looking at the byte at offset zero. You don’t have to know how long the number is, nor do you have to skip over any bytes to find the byte containing the sign information. The numbers are also stored in the order in which they are printed out, so binary to decimal routines are particularly efficient.

Ex :

1024 (0×00000400) stored in little endian machine

as

00 00 04 00

in big endian machine stored as

00 04 00 00

is it possible to find endianness at compile time?

no, it is not possible. but we can use compile time flags

ex.

#ifdef _BIG_ENDIAN

//this code

#else

//this code

#endif

How to find endianness by a program

int x=1 ;if(*(char *) &x == 1)

printf("little endian") ;

else

pritf("big endian") ;

[tags]big endian, little endian, endianness, embedded system programming[/tags]

Byte order utililities

Previous post little, big endianness explained deals with theoretical part of endianness, now let us write some utility functions for endainness.

Little Endian To Host: Data we are taking here is little endian data and this program converts little endian to host format. program takes a buffer of data and rearranges it to the native order of the machine. if the native order is little endian no conversion happens.

IsLittleEndian function is already listed in previous post, as it is used here, we?ve copied that function?

bool IsLittleEndian()
{
int x=1 ;
if(*(char *) &x == 1)
return true;
else
return false;
}

int LittleEndianToNative(char *data, unsigned int size)
{
char tmp;
char *sptr, *eptr;

if (!data || !size)
{
return -1;
}

if (IsLittleEndian())
{
return 0;
}

sptr = data;
eptr = data + size - 1;

while (sptr <= eptr-1)
{
tmp = *sptr;
*sptr++ = *eptr;
*eptr– = tmp;
}
return 0;
}

Host to Little Endian:

Data we are taking here is host or native order data and this program converts host to little endian format. program takes a buffer of data and rearranges it to little endian. if the native order is little endian no conversion happens.

int NativeToLittleEndian(char *data, unsigned int size)
{
//conversion process is same
return LittleEndianToNative(data, size);
}

Big Endian To Host:

Data we are taking here is big endian data and this program converts big endian to native order format. program takes a buffer of data and rearranges it to the native order of the machine. if the native order is big endian no conversion happens.

int BigEndianToNative(char *data, unsigned int size)
{
char tmp;
char *sptr, *eptr;

if (!data || !size) { return -1; }

if (!IsLittleEndian())
{
//no conversion required
return 0;
}
sptr = data;
eptr = data + size - 1;

while (sptr <= eptr-1)
{
tmp = *sptr;
*sptr++ = *eptr;
*eptr– = tmp;
}
return 0;
}

Host to Big Endian:

Data we are taking here is host or native order data and this program converts host to big endian format. program takes a buffer of data and rearranges it to big endian. if the native order is big endian no conversion happens.

int NativeToBigEndian(char *data, unsigned int size)
{
//conversion process is same
return BigEndianToNative(data, size);
}

sections of a program

2009-08-31T19:18:00.000-07:00

-> volatile, static, const, register vars
-> Sections/Segments (text, data, stack, heap, bss, etc) of an ELF file
-> where various variables stored
local, formal vars: stack
static,global (unintialised): bss (block start by symbol)
static, global(initialised) : data section
vars allocated thru malloc, new: heap

-> static linking (.a), dynamic linking and loading (.so)
-> Bitwise operations
-> swap two vars without using tmp var
a=a+b; b=a-b; b= a-b;
u can also swap with using xor operations
a^=b^=a^=b? (ofcourse this doesn’t work always as u expected, check the sequence point.. modify it)
-> calling conventions
info: c-calling convention, standard calling convention, pascal. etc. (this stuff specific windows,
unix don’t bother about this, it just uses c calling convention)

C++

===
-> Difference between structure and class
-> access specifiers
-> Whats the size of an empty class
-> When do you use composition/aggregation and inheritance
-> When RTTI useful?
-> inline, macros
-> virtual polymorphism (abt vtables, vptrs also)
-> virtual destructors, why not virtual constructors
-> If u invoke a virtual function in constructor what will happen
-> How static behave in C++
-> Name mangling (all related issues like function overloading, using a C or other language function, etc)
-> oop concepts (just abstraction, encapsulation, inheritance, polymorphism)
-> copy constructors
-> “delete this”… is it advisable to use like this? is it useful in any situation?
-> what happens if an exception occurs in between construction of an object
-> If I link a library which I compiled on solaris with a linux executable, will it work?
why?
-> difference between #define constants and enums
-> class
{
enum x{one, two}X1;
enum y{one, two}Y1;
};
is this is OK?

Programming
===========

-> Write a program to search a list of unsorted elements with only one condition
Ans:
Normal routine: for(i=0;i Manage an array efficiently to implement two stacks on it
Ans:
manage like this:

————————————
|1st stack–> <–2nd stack|

————————————-

no explicit bounds

Write a program to traverse a n-ary tree and store the nodes in disk in such way that
tree is reconstructable from the storage

Ans: like java’s serialise data structures

-> Write a program to traverse an array spirally
Ans: write this using directions and maintaining states

-> Write a program to find sub-array of any number of elements with optimum
sum-value in an array (array elements may be -ve)

-> Reverse a double linked list

-> Reverse a single linked list
Ans: use a stack to reverse it

-> Write a recursive program to print a linked list reversely

print(Node *node)
{
if(node->next)
print(node->next)
else
return;
printf(node->info)
}

-> How do you find loops in a single linked list
1) use flags
2) use two or three pointers and increment them at different levels
like one ptr with ptr=ptr->next , another with second_ptr=second_ptr->next->next.
if these pointers meet, then it got loops

-> Write a program to find x^n in O(log n)

xpown(x,n)
{
val = x;

do {
if(n % 2)
val *= x;
else
val *= val;
}while(n /= 2);
}

-> Write a program for preorder traversal without using recursion
ans: use stack

-> Write a program to traverse a binary tree in level by level manner
ans: its just BFS… use queue

-> finding a number is power of 2 or not
ans:
if(!(n & n-1)) printf(”power of 2″);
better answer like this “count number of 1’s… if(count==1) power of 2)

->In an unsorted array find the sum of 3 maximum numbers in less than O(n) time
ans: 1)do bubble sort until u sort 3 elemnts then stop it, sum the nums
2) build heap (O(n)), delete_max 3 elements (O(logn))

-> in a sorted array only two elements are misplaced, find the desired element still
maintaining the o(logn) time

-> basics of all data structres

Structure

2009-08-31T15:14:00.002-07:00

5. DATA STRUCTURES
5.1 Data Structure

A collection of the same type of objects is called a data structure.
5.2 Linked List

If you want to maintain a collection of the same objects, you may first think of an array of that type of objects. But as we already know, the size of the array have to be decided before compiling. Then we may think of using dynamic memory allocation to create the array, so that we can decide the memory size at run time:

Test * array = new Test [size];

But still there is limitation: once the array is created, you can not vary the size, unless you delete the whole array and create a new one. You can’t expand or shrink the list dynamically.

In order to put a group of objects into a flexible data structure, you can use wrapper objects to wrap around the original objects, with some additional pointers pointing to each other. Through these pointers, all the wrapper objects can be linked together. Such a pointer-linked collection is called “linked list”.

The disadvantages of linked list is that you can not directly access one node with subscript like array. You have to find the first one, go through the list through the pointers, until you reach the one you want. The other thing to remember is: nodes in linked list are created dynamically. They are not as fast as automatic objects.

If in a data structure each of the nodes is only connected to one node downstream and one node upstream, such a structure is called a “linear linked list”.

There are “open-loop” lists, each has a start node and end node. The pointer of the end node is always set to 0 to mark the end. There are also “close-loop” lists, whose start node and end node are connected together forming a circular loop of nodes.

There are Singly-linked lists whose nodes only contain one pointer pointing to the downstream node. You can’t traverse upstream. There are also doubly-linked lists whose nodes each contains a pointer to both the upstream and downstream node. You can traverse on both directions.

Using pointers to link objects you can create lists as queues, which only removes a node from the front and adds a node to the back, or stacks, which only adds and removes nodes to and from the front, or any other types.
5.3 Tree

A tree is a structure whose nodes each connected to one upstream node and many down-stream nodes. A binary tree is a structure which is connected to one upstream and two downstream nodes. A tree has a root node.
5.4 Struct

A structure is a simplified class with all public members:

struct Card
{
char * face;
char * suit;
};

Structure members are not necessarily stored in consecutive bytes of memory. If the word length of the computer is 2 bytes, and one data member is only one-byte long, there will be a one-byte hole between this member and the next member. Because the value stored in the hole is undefined, a structure can not be compared.

A structure is initialized the same way as an array:

Card c1 = {"Three", "Hearts"};

If the initialization list contains not enough initializers, the rest will be initialized to 0.
5.5 Bitwise Operators

Up to now data is regarded as a whole number without considering its bit status. Internally a number is represented by binary bits. C++ provides the following bitwise operators:

1)&: bitwise AND. One bit is 1 if the corresponding bits of the both operands are both 1.
2)|: bitwise inclusive OR. One bit is 1 if one of both corresponding bits of the two operands is/are 1.
3)^: bitwise exclusive OR. One bit is 1 if the corresponding bits of the two operands are different: one is 1 and one is 0
4)<<: left shift. Shifts the bits of the first operand left by the number of bits specified by the second operand; fill from right with 0. For example, 0001 << 2 will be 0100. Result will be undefined if the second operand is negative.
5)>>: right shift. Shifts the bits of the first operand right by the number of bits specified by the second operand; The way of filling the left is machine-dependent, some use 0 and some use the sign bit.
6)~: complement. All 0 bits are 1 and all 1 bits are 0. For example, ~0101 will be 1010.
7)Assignment operators: &=, |=, ^=, <<=, >>=. For example: a &= b means a = a & b, a >>= b means a = a >> b.
5.6 Example: Display a Number in Binary Form

void BinaryDisplay1(unsigned value)
{
mask = 1 << 15; // 1000000000000000

for (int c = 1; c <= 16; c++)
{
// 1000000000000000 is regarded as true
cout << (value & mask ? '1': '0');

if(c % 8 == 0)
cout << ' ';
}

cout << endl;
}

5.7 Bit Field

C++ provides the ability to specify the number of bits in which an unsigned or int member of a class or a structure (or a union) is stored. Such a member is called a bit filed member:

struct Card
{
unsigned face : 4;
unsigned suit : 2;
unsigned color: 1;
};

Therefore, three members are stored in one byte. The number of bits of one member is decided by the maximum number it will represent. For example, the number of faces of cards is 13, less than 16, so member “face” can be defined with four bits.

Bit field members are accessed the same way as normal members. But they can not have addresses, and “&” can not be used, because there is no unique address of each part of one word.

Unnamed bit field with a width is used as padding, nothing can be stored in these padding bits:

structure Example
{
unsigned a : 13;
unsigned : 3;
unsigned b : 4;
};

Unnamed bit field with 0 width is used to align the next bit field with next word:

structure Example
{
unsigned a : 13;
unsigned : 0;
unsigned b : 4;
};

Although bit fields save space, it consumes more machine time, because extra codes have to be generated to deal with them. This is one of the many examples of space-time trade off that occur in computer science.
5.8 Character Handling Library

Characters are often manipulated as integers. Header file of the character handling library is .

int isdigit(int c): returns true if c is a digit — ‘0′ to ‘9′.
int isxdigit(int c): returns true if c is a hex digit — ‘0′, ‘1′,…, ‘E’, ‘F’.
int isalpha(int c): returns true if c is a letter.
int isalnum(int c): returns true if c is a letter or a digit.
int islower(int c): returns true if c is a lower case
int isupper(int c): returns true if c is a upper case
int tolower(int c): if c is upper case then return c in lower case, otherwise return c unchanged
int toupper(int c): if c is lower case then return c in upper case, otherwise return c unchanged.
int isspace(int c): returns true if c is a whitespace character: newline(’\n’), space, form feed(’\f’), carriage return(’\r’), horizontal tab(’\t’), or vertical tab(’\v’).
int iscntrl(int c): returns true if c is a control character.
int ispunct(int c): returns true if c is a printing character other than a space, a digit or a letter.
int isprint(int c): returns true if c is a printing character including space.
int isgraph(int c): returns true if c is a printing character other than space.
5.9 String Conversion Methods

String conversion methods are in .

double atof(const char *): converts a numbered string to double. E.g.: if ptr = “23.85″, double 23.85 will be returned.
int atoi(const char *): converts a numbered string to integer. E.g.: if ptr = “2593″, integer 2593 will be returned.
long atol(const char *): converts a numbered string to long integer.

double strtod(const char *, char **): converts the double portion of string ptr1 to double, and locate the second pointer at the first remaining character in the first string. E.g.: if ptr1 = “1234.2ABCD”, double 1234.2 will be returned and second pointer will be “ABCD”.

long strtol(const char * ptr1, char ** ptr2, int base): converts the long int portion of string ptr1 to long integer, and locate pointer ptr2 at the first remaining character in ptr1. The long integer can be in any base: if base = 0, then it can be decimal, octal or hexadecimal; if base = 8, it is octal; if base = 10, it is decimal; if base = 16, it is hexadecimal. But the base can be any value from 2 to 36, limited only by the number of Latin letters A-Z. Using NULL for the second argument will cause the remaining portion be ignored.

unsigned long strtoul(const char * ptr1, char ** ptr2, int base): only difference with method strtol is the integer is unsigned.
5.10 String Handling Library “string.h”
=>Searching character in string

char * strchr(const char * s, int c): locate the first c in string s, or return NULL if not found.
size_t strcspn(const char * s1, const char * s2): return the length of the initial segment of s1 which consists of characters not included in s2.
size_t strspn(const char * s1, const char * s2): return the length of the initial segment of string s1 which consists only characters in s2.
char * strpbrk(const char * s1, const char * s2): locate the first occurrence in s1 of any character contained in s2 or NULL if not found.
char * strrchr(const char * s, int c): locate the last occurrence of c in s or NULL if not found.
char * strstr(const char * s1, const char * s2): locate the first occurrence in s1 of s2 or NULL if not found.
=>Memory manipulating methods

The basic unit of memory is byte, and a byte can be regarded as a character. The following methods manipulate memory buffers in characters. The pointer parameters in these methods are declared “void *”. Because pointers of any type can be directly assigned to a pointer of type “void *”, therefore methods can receive pointers of any type.

void * memcpy(void * s1, const void * s2, size_t n)
Copies n characters from buffer s2 to s1, and return a pointer to the resulting buffer.

void * memmove(void * s1, const void * s2, size_t n)
The same as memcpy, but the characters are first copied from s2 into a temporary location, then copied to s1. This allows to copy part of one object and overlap with itself:

int main()
{
char x[] = "123 456 789";
cout << (char *)memmove(x, x+4, 3);
}

then x will become “456 456 789″.

int memcmp(const void * s1, const void s2, size_t n)
//Compares the first n characters of object s1 and s2,
//and returns 0, <0, or >0.

void * memchr(const void * s, int c, size_t n)
//Locates the first occurrence of c in the first n
//characters of object s, or return 0 if not found.

void * memset(void * s, int c, size_t n)
//Set the first n characters of object s to character c,
//and return a pointer to the result

int main()
{
char x[] = "AAAA";
cout << (char *)memset(x, 'B', 2);
}

result will be “BBAA”.
5.11 Error Message Method

char * strerror(int code);

Error messages have their codes. This method maps the error code into a full text string in a system dependent way, and returns a pointer to that mapped message.

unnamed bit field

2009-08-31T15:14:00.001-07:00

An unnamed bit field is used for padding to conform to external imposed layouts.

Unnamed bit-field with a width of zero specifies alignment of the next bit field at an allocation unit boundary.

struct becks {
…
int :3 //three bit field
…
}

Padding the the significance!

2009-08-31T15:12:00.001-07:00

What is Padding?
Aligning members of a structre to a byte boundary. Padding is done by compiler to gain the performance. Most hardware architectures
access addresses fast if they are aligned properly, otherwise there will be performance penalty. In order to increase performance compiler pads structures.

How Padding done?
Normally processors require members of the structure aligned to byte boundary, for instance char member(1 byte) should be aligned to
one byte boundary that is it can appear at any byte boundary, a short int(2 bytes) should be aligned to 2 byte boundary that is it should appear at 2 byte boundary, long int(4 bytes) should be aligned to 4 byte boundary. If we declare a basic data type in structure it should appear at sizeof(basic data type) byte boundary, in order to make the start address of the next member at an aligned address.

Example:
Struct X
{
Char a;
Short int b;
Char c;
Long
d;
};

| a(1) | P(1)| b(2) | c(1)| P(3) | d(4) |

P - padding bytes

Why Padding?
If the processor is serious about alignment it raises an exception/signal (ex. SIGBUS), otherwise there will be a performance penalty
will be there as misalignment slowdown data access
.

Are members order going to be changed?
ANSI C requires fields of a structure need to allocated in the order they are declared.

As a programmer how does padding matter to me?
It matters if you are concerned about space it takes. By changing the members order of declarations, some amount space can be saved. For example
Struct X
{
Char a;
Long
Int b;
Char c;
Long int d;
Char e;
Long int f;

};

Sizeof(struct X) = 24 bytes

Lets reorder the above structure as below

Struct X
{
Char a;
Char c;
Char e;
Long Int b;
Long int
d;
Long int f;
};

Sizeof(struct X) = 16 bytes

Padding is only done to members or to the structure as a whole
also?
Padding will be done by compiler to structure’s members and to the structure as a whole also. Compiler pads structure as whole because this allows each member of structure aligned in array of structures.

Packed structures
Some compilers provide #pragma to suppress padding or to make it packed to n number of bytes. Some provide keywords to do this.
Generally pragma which is used for modifying structure padding will be in the below format (depends on compiler)
#pragma pack(n)
For example arm
provides __packed keyword to suppress structure padding. Go through your compiler manual to know more about this.

So a packed structure is a structure without padding.
Generally packed structures will be used
->to save space, and
->to format a data structure to transmit over network using some protocol (this is not a good practice of course because you need to deal
with endianness)

How to declare array of packed structures?
We can not declare array of packed structures as for declaring as an array we need each member of structure aligned. So alternatively either we can declare something like this
Struct X
{
Unsigned char a;
Unsigned long b;
}array[NUM_ELEMENTS];

The above structure yields compilation error, if we declare this as below, we should be able to handle..
Unsigned char array_bytes[sizeof(structure) * NUM_ELEMENTS];
And take special care while handling array_bytes.

When I need to be careful about structure padding?
1)when writing structures to a file on one machine and reading them on different machine
This involves problem, consider this example
Struct X
{
Char a;
Long int b;
};

x.a = 0�01;
x.b = 0�02;

If you write struct X to a file, it would have written 0�01 G G G 0�00 0�00 0�00 0�02 (G is garbled value, because this area is padded)
While reading this on different machine(could be big endian) we read the structure as whole, on big-endian these values will be altered and as a result we read corrupted value for member a;
Solution for this would be using a packed structures we can eliminate this.

How can I pack structures Using GCC?
Using the "packed" attribute against the members of a structure. This attribute mechanism is an extension to the GNU C compiler. An example of how you would do this is below.

struct X
{
unsigned char a __attribute__((__packed__));
unsigned
short b __attribute__((__packed__));
unsigned long c
__attribute__((__packed__));
};

Another way to achieve the same is as shown below:
struct X
{
unsigned char a;
unsigned short b;
unsigned long c;
}
__attribute__((__packed__));

How can I pack structures Using ARM compiler?
__packed keyword is used for this purpose, usage is shown below

__packed struct X
{
char a;
Long int b;
};

May be in some other post, I try to give details about how padding done for
global variables, pointers and etc. on some selected machines. Your comments,
suggestions, criticism, etc. most welcome.

link list

2009-08-30T18:37:00.001-07:00

#include
#include
struct link_list{
int data;
struct link_list *next;
};

void insert(int data, struct link_list **usrlist){
struct link_list *temp, *trav;
temp=(struct link_list *)malloc(sizeof(struct link_list));
temp->data=data;
temp->next=NULL;
if(*usrlist==NULL){
*usrlist=temp;
}
else{
trav=*usrlist;
while(trav->next){
trav=trav->next;
}
trav->next=temp;

}
}

void display(struct link_list **usrlist){
struct link_list *trav=*usrlist;
printf("\n");
if(trav==NULL)
printf("\nThe list is empty\n");
while(trav){
printf("data: %d -->",trav->data);
trav=trav->next;
}
}

struct link_list * revlist(struct link_list **userlist){
struct link_list *temp, *usrlist,*prev=NULL;
usrlist=*userlist;
while(usrlist->next){
temp=usrlist->next;
usrlist->next=prev;
prev=usrlist;
usrlist=temp;
}
usrlist->next=prev;
return usrlist;
}

struct link_list * freelist(struct link_list **usrlist){
struct link_list *temp;
while(*usrlist){
temp=*usrlist;
*usrlist=(*usrlist)->next;
free(temp);
printf("\nEntry freed");
}
return NULL;
}

struct link_list * revlistrec(struct link_list *L)
{
struct link_list *first, *rest, *trav, *reftrav;
first=L;
rest=L->next;
if(!L)
return NULL;
if(!L->next)
{
return L;
}
if(!L->next->next)
{
L->next->next=first;
first->next=NULL;
return rest;
}
reftrav=trav=revlistrec(rest);
while(reftrav->next)
{
reftrav=reftrav->next;
}
reftrav->next=first;
first->next=NULL;
return trav;

}

void sortlist(struct link_list *L){
struct link_list *temp;
int datatemp;
while(L){
temp=L;
while(temp){
if(L->data>temp->data){
datatemp=temp->data;
temp->data=L->data;
L->data=datatemp;
}
temp=temp->next;
}
L=L->next;

}
}

void middlelist(struct link_list *L){
struct link_list *second;
second=L;
while(L->next!=NULL && second->next!=NULL && second->next->next!=NULL){
L=L->next;
second=second->next->next;
}
printf("\n the middle element is %d", L->data);
}

void main()
{
struct link_list *usrlist=NULL;

if(usrlist==NULL){
printf("\nThe users list is empty...i am inserting data");
insert(20, &usrlist);
insert(2, &usrlist);
insert(33, &usrlist);
insert(1, &usrlist);
insert(2, &usrlist);
insert(63, &usrlist);
insert(42, &usrlist);
insert(90, &usrlist);
insert(91, &usrlist);
}
display(&usrlist);
//usrlist=revlist(&usrlist);
//usrlist=revlistrec(usrlist);
//printf("\n\nreversed display");
printf("\nmiddledisplay");
// sortlist(usrlist);
middlelist(usrlist);
display(&usrlist);
printf("\n Insertion and display finished\n");
usrlist=freelist(&usrlist);
display(&usrlist);
}

use of double pointers in structures

2009-08-30T15:04:00.000-07:00

No. The reason for the pointer to a pointer parameter is so that the result of malloc could be reflected in the caller (i.e., if you just passed a pointer to a struct node, the result of malloc would just be assigned to the local struct node pointer rather than the struct node pointer from the caller). If you do not need such a call by reference parameter (which is the case for a print or a search function), then just passing a pointer to a struct node will do.

Rule 1 of software engineering

2009-08-28T21:21:00.000-07:00

The amount of time trying to figure out a solution to the bug is directly proportional to the amount of stupidity involved in introducing it at the first place

----- i was told about it by Viji but I dont know who gave the original version....

Other laws by viji:

A bug is constant, it migrates from one code to another.

installing wireless using windows driver

2009-08-07T13:27:00.000-07:00

OWTO: Dell Inspiron Wireless (Broadcom 1390 WLAN)
This HOWTO describes how to get Wifi working on your Dell Inspiron E1505/6400 laptop using Ndiswrapper. This applies if you have the Broadcom "Dell Wireless 1390 WLAN MiniCard", which as far as I know, is the cheaper, low-end version that Dell currently offers in this laptop. This HOWTO has been tested and works with both 32 bit and 64 bit ubuntu, and uses the exact same method for both. It is VERY IMPORTANT that you read every part of this how-to and follow the steps EXACTLY as they're written. Many users have trouble because they only read the code sections and ignore my directions to change to the proper directories. Again, PLEASE BE SURE YOU ARE IN THE PROPER DIRECTORY WHEN EXECUTING THESE COMMANDS. This will save you a lot of trouble and will make the installation go a lot smoother.

This HOWTO supports:

* Gutsy Gibbon 7.10
* Feisty Fawn 7.04
* Edgy Eft 6.10
* Dapper Drake 6.06

STEP 1: CLEAN YOUR SYSTEM

IMPORTANT NOTE ABOUT CLEANING YOUR SYSTEM:

One of the most common reasons why many people can't get their wireless working is because their system is in a state of chaos. If you have made ANY previous attempts to get your wireless working -- either using fwcutter, ndiswrapper, or the bcm43xx drivers -- this how-to will most likely not work UNTIL you reverse your previous changes. In many cases, it is much easier to simply reinstall ubuntu and come straight to this how-to. Alternatively, you can manually clean your system of the previous attempts, as outlined in various posts throughout this thread. But BE WARNED: If you have done ANY previous work on your wireless, there is almost no chance that this how-to will work unless you clean your system.

If you have a fresh install of Ubuntu, you need to remove any and all versions of Ndiswrapper that come installed by default on your system:

Code:

sudo rmmod ndiswrapper
sudo ndiswrapper -e bcmwl5
sudo apt-get remove ndiswrapper-utils

Don't worry if you get errors about not being able to find or remove these -- we're just making sure they're not present before we get started.

STEP 2: GET NEEDED PACKAGES

We'll need to install compiling tools (don't panic when you read that, just bear with me), the latest kernel headers, and then the source code for the latest ndiswrapper (seriously, don't panic. This will be very simple), and the wireless drivers from Dell.com.

Code:

sudo apt-get update
sudo apt-get install build-essential
sudo apt-get install linux-headers-`uname -r`
wget http://ftp.us.dell.com/network/R151517.EXE

NOTE: The characters around `uname -r` are BACK TICS, NOT apostrophes. A back tic is usually located at the top left of your keyboard, to the left of the 1 key. The command WILL NOT WORK if you use apostrophes. Just copy/paste the commands from this how-to in to your terminal to avoid making typos.

At this point, you need to go to the ndiswrapper sourceforge site and get the latest version of the Ndiswrapper program.

Code:

wget http://superb-east.dl.sourceforge.net/sourceforge/ndiswrapper/ndiswrapper-1.51.tar.gz

If that wget doesn't work, just go here: http://sourceforge.net/project/showf...group_id=93482

Uncompress the ndiswrapper source (in my example, the file name is ndiswrapper-1.51.tar.gz):

Code:

tar -xzvf ndiswrapper-1.51.tar.gz

Finally, we need to blacklist the broken and useless bcm43xx firmware drivers that try to load in a default ubuntu install:

Code:

sudo echo blacklist bcm43xx >> /etc/modprobe.d/blacklist

NOTE: If the above command gives you a permission denied error, try this code instead:

Code:

sudo -s
echo blacklist bcm43xx >> /etc/modprobe.d/blacklist
exit

YOU MUST REBOOT NOW!

STEP 3: COMPILE PROGRAM

Now we'll complile the Ndiswrapper program. In a terminal, go to the directory where you extracted ndiswrapper and execute the following:

Code:

cd YOUR-NDISWRAPPER-DIRECTORY
sudo make uninstall

IMPORTANT: Do the above command multiple times. You can stop when you get the message that says something about no files or directories found. This usually means running the command 2 or 3 times, but not more than about a dozen.

Code:

sudo make distclean
sudo make
sudo make install

STEP 4: INSTALL DRIVERS

If that worked, then you now have Ndiswrapper installed. Now we need to install the drivers. In a terminal, go to the directory where you have the R151517.EXE file:

Code:

unzip -a R151517.EXE

Now change directories to the DRIVER directory that was just extracted.

Code:

cd YOUR-DRIVER-DIRECTORY
sudo ndiswrapper -i bcmwl5.inf
sudo ndiswrapper -l

you should see a message that says driver present, hardware detected

Code:
NDISwrapper in Fedora using Dell TrueMobile 1300
Submitted by sandip on Sat, 10/09/2004 - 12:35. | Tags:

* Tutorials

0
points

This step-by-step walk through is for Fedora CORE 1.0 using a Dell Latitude CPi with the PCMCIA Dell TrueMobile 1300 (BroadCom BCM94306) 802.11b/g wireless network card.

Install the ndiswrapper by following the installation wiki documentation.

NOTE: I was getting errors with the current release and 0.9 version finally worked for me, so try installing previous releases and use the one which works for you.

Below are the steps that I took:
1. Install the ndiswrapper using the rpm.

#tar -zxvf ndiswrapper.tar.gz
#cd ndiswrapper
#make rpm
#rpm -Uvh /usr/src/redhat/RPMS/i386/ndiswrapper*.rpm

2. Install the windows driver as described in the infFile using ndiswrapper. You should have a ".inf" and a ".sys" file for this to work.

#mount /mnt/cdrom (place the Dell TrueMobile Driver disk in the cdrom first)
#ndiswrapper -i /mnt/cdrom/Setup/bcmwl5.inf

3. Load the ndiswrapper module

#modprobe ndiswrapper

The light on the card should turn on at this point.

4. Create and setup the network parameters for the interface wlan0.

#cat < /etc/sysconfig/network-scripts/ifcfg-wlan0
>DEVICE=wlan0
>BOOTPROTO=dhcp
>ONBOOT=yes
>GATEWAY=192.168.2.1
>TYPE=Wireless
>ESSID=
>CHANNEL=
>MODE=Managed
>KEY=
>EOF

5. Configure the wireless network interface

#iwconfig (look for your wlan card, mine came up with wlan0)
#iwconfig wlan0 mode Managed
#iwconfig wlan0 key
#iwconifg wlan0 essid
#ifconfig wlan0
#route add default gw <192.168.2.1>
#ifup wlan0

I got an error but then it said '... done'.

6. Check the wireless interface is up.

#ifconfig (should list out wlan0 with an inet address)
#ping 192.168.2.1 (Voila success!!)

When this worked I instantly went to linuxweblog.com and noted down these steps for reference.

7. Instead of automating the modprobe settings to load ndiswrapper as mentioned in the install docs via `ndiswrapper -m`, I put `modprobe ndiswrapper` in the "rc.local" file.

#cat <> /etc/rc.d/rc.local
># Load ndiswrapper module
>modprobe ndiswrapper
>EOF

sudo ndiswrapper -m
sudo modprobe ndiswrapper
sudo echo ndiswrapper >> /etc/modules

NOTE: If the above echo command gives you a permission denied error, try this code instead:

Code:

sudo -s
echo ndiswrapper >> /etc/modules
exit

Some users have reported the need to reboot here.

STEP 5: TEST WIRELESS

Your wifi light on your laptop should be illuminated, and you're all set! Try running this to see if your wireless card is functioning properly:

Code:

sudo iwlist scanning

Even if it doesn't detect any wireless networks in range, it will still tell you if linux is recognizing your wireless card properly. If you'd like a better way to scan for wireless networks, I'd suggest installing/using network-manager or wifi-radar.

TO ALL WHO REQUEST HELP:

1) Please indicate:

1. your laptop model
2. ubuntu version
3. your lspci output
4. the relevant rows from /var/log/syslog
5. the relevant rows from dmesg
6. any unexpected output when following the steps in the how-to

2) If your wireless card is being recognized by the system, but you simply cannot connect to your desired wireless network, please understand that your issue is beyond the scope of this how-to, due largely to the fact that such issues are very complicated and often have absolutely nothing to do with the way you got ubuntu to recognize your wireless card. That said, I will do what I can to help. My first suggestion will always be to turn off all encryption first to see if you can at least connect to an open network.

NOTES:

- Dell driver version updated to R151517.EXE on 30 March 2007
- Ndiswrapper version updated to 1.51 on 7 January 2008
- If you are using a Dell laptop that's NOT model E1505, you need to go to Dell.com and search for the drivers that correspond to your specific model. Use those instead.

***********************************************************************************************
NDISwrapper in Fedora using Dell TrueMobile 1300
Submitted by sandip on Sat, 10/09/2004 - 12:35. | Tags:

* Tutorials

0
points

This step-by-step walk through is for Fedora CORE 1.0 using a Dell Latitude CPi with the PCMCIA Dell TrueMobile 1300 (BroadCom BCM94306) 802.11b/g wireless network card.

Install the ndiswrapper by following the installation wiki documentation.

NOTE: I was getting errors with the current release and 0.9 version finally worked for me, so try installing previous releases and use the one which works for you.

Below are the steps that I took:
1. Install the ndiswrapper using the rpm.

#tar -zxvf ndiswrapper.tar.gz
#cd ndiswrapper
#make rpm
#rpm -Uvh /usr/src/redhat/RPMS/i386/ndiswrapper*.rpm

2. Install the windows driver as described in the infFile using ndiswrapper. You should have a ".inf" and a ".sys" file for this to work.

#mount /mnt/cdrom (place the Dell TrueMobile Driver disk in the cdrom first)
#ndiswrapper -i /mnt/cdrom/Setup/bcmwl5.inf

3. Load the ndiswrapper module

#modprobe ndiswrapper

The light on the card should turn on at this point.

4. Create and setup the network parameters for the interface wlan0.

#cat < /etc/sysconfig/network-scripts/ifcfg-wlan0
>DEVICE=wlan0
>BOOTPROTO=dhcp
>ONBOOT=yes
>GATEWAY=192.168.2.1
>TYPE=Wireless
>ESSID=
>CHANNEL=
>MODE=Managed
>KEY=
>EOF

5. Configure the wireless network interface

#iwconfig (look for your wlan card, mine came up with wlan0)
#iwconfig wlan0 mode Managed
#iwconfig wlan0 key
#iwconifg wlan0 essid
#ifconfig wlan0
#route add default gw <192.168.2.1>
#ifup wlan0

I got an error but then it said '... done'.

6. Check the wireless interface is up.

#ifconfig (should list out wlan0 with an inet address)
#ping 192.168.2.1 (Voila success!!)

When this worked I instantly went to linuxweblog.com and noted down these steps for reference.

7. Instead of automating the modprobe settings to load ndiswrapper as mentioned in the install docs via `ndiswrapper -m`, I put `modprobe ndiswrapper` in the "rc.local" file.

#cat <> /etc/rc.d/rc.local
># Load ndiswrapper module
>modprobe ndiswrapper
>EOF

hack the crack :D something i discovered during the last years of my undergrad!

2009-07-23T09:35:00.001-07:00

We will have a very basic and simple approach. We will use a disassemble and an assembler.

Disassemble is the process that will let us to view the "asm" source code of the
dissembled file.
Assembler - is the process that will allow us to make changes to the code.

The most classic disassemble is W32Dasm, download it here:
CODE
http://foff.astalavista.ms/downloads/W32Dasm_8.93.zip

The best assembler is HIEW32, download it here:
CODE
http://foff.astalavista.ms/downloads/Hiew726w.zip

Download the file that we will crack here:
CODE
http://foff.astalavista.ms/downloads/cim_crackme.zip

Unzip everything and we are ready to start!!!

Step1.

Run the file "cim_crackme.exe", we see there two empty boxes. Name and Serial like many
shareware software. Put there your name and any serial number then click on "Check"
button. We will receive this message: "This is not a valid serial......" Write this
message down, don't make errors. Just write the message without the quotes and the dots of course then close everything.

Step2.

Run W32Dasm disassemble, on the menu bar open the first menu "Disassemble" the "Open
file to disassemble". Browse to our target "cim_crackme.exe" then disassemble. You now
should see a lot of code on your screen

Go at the search menu of W32Dasm and click on "Find Text", put in the search box the text
string we did memorize somewhere: "this is not a valid serial" (without the quotes).

:

*Referenced by (U)nconditional or ©onditional Jump at Address:
|:004010D7

You should record the address you see: 004010D7 (is very important)

Now, go up once more until you find the address you just recorded,

The arrow number 1 show who is addresses group located. In the circle is the address we
were looking for. The address and all the row is selected by the green bar too! I hope
everything is clear enough! As you see, there is an other arrow, "arrow 2" that indicates
"JNE". What does "JNE" mean???

JNE - Jump if not equal
JE - Jump if equal

When we enter a fake serial number, it jumps. Hmmm, we should reverse it. The reverse of
"JNE" is "JE". If we do this, the program will accept any serial number as a real one!!!

But how do we edit it???

Using an assembler of course

Our mission now is to reverse "JNE" to "JE" at the address: "004010D7".

Step3.

Now send a desktop icon of HIEW32 then drag and drop "cim_crackme.exe" over it. You should
see some crap code, now press F4 from the keyboard then choose decode from the selection.

Now, return to HIEW and press F5 from keyboard then enter the address we recorded above
but don't forget to add a dot before the address and to remove the zeros before the
address number. Enter it like this: ".4010D7" (without the quotes of course) then hit
ENTER. After this you will land in the exact address where we will do the reversing.
Notice this line carefully:

.004010D7: 7516 jne .0004010EF

We should change the bytes here. Without moving the selection from "75" press F3 from
keyboard then hit the right arrow of the keyboard directional keys one time, be sure to
have the cursor under the number "5". Carefully press the number 4 from the keyboard. The
number will change from "7516" to "7416". Carefully press F9 from keyboard to save our
changes and F10 to exit at all! We are done!!!

Now check that everything is ok, run "cim_crackme.exe" that you just cracked and put you
name and any serial number..............CONGRATULATIONS!!! You Cracked It.

MBR and its back up

2009-05-13T16:32:00.000-07:00

1. Use the Norton Utilities.

2. Use Debug, as found in DOS, it is also available from the DOS prompt
in Windows/95 so I assume it is also available in Windows/98.

Enter the following commands to save the MBR on the C drive, ignore
the text after the '<=' on each line as it is only a comment:

DEBUG MBR.DAT <= Ignore the FILE NOT FOUND message
A <= Assemble a program
MOV DX,9000 <= Use segment 9000
MOV ES,DX <= Setup the segment register
XOR BX,BX <= Clear BX to zero
MOV CX,0001 <= Start at track 00 sector 01, the MBR
MOV DX,0080 <= 80=C:, 81=D:, 00=A:, 01=B:
MOV AX,0201 <= Read 1 sector, 02=read
INT 13 <= BIOS disk i/o call
INT 20 <= Return to o/s
<= Press the return key to end program entry
G <= Execute the program
R CX <= Display the value of CX
:200 <= Change the value of CX to decimal 512, size of
MBR
W 9000:00 <= Write the sector stored at address 9000 to
MBR.DAT
Q <= exit DEBUG

If you examine the contents of MBR.DAT using a disk file editor the last

2 bytes must be AA55.

At this point you should copy the MBR to a bootable floppy along with
DEBUG.EXE

This technique may be used to recover the MBR as well, assuming you can
boot from another device (say a floppy with DEBUG.EXE on it).

Enter the following commands to restore the MBR on the C drive, ignore
the text after the '<=' on each line as it is only a comment:

DEBUG MBR.DAT <= The file containing the desired MBR, if you
get a
FILE NOT FOUND message type Q immediately! If
you
continue you will write garbage over the MBR.
L 9000:00 <= Load the MBR into memory at this address
A <= Assemble a program
MOV DX,9000 <= The segment address containing the MBR
MOV ES,DX <= Setup the segment address
XOR BX,BX
MOV CX,0001 <= Track 00, sector 01
MOV DX,0080 <= 80=C:
MOV AX,0301 <= Write one sector, 03=write
INT 13
INT 20
<= Press the enter key to stop program entry
G <= Execute the program
Q <= Exit DEBUG

The MBR should now be restored to the C drive, making it bootable.

For more information on this technique for saving and restoring an MBR I

refer you the book 'The Complete PC Upgrade and Maintenance Guide' by
Mark Minasi, published by Sybex.

Once you have Linux running you can save the boot record with the
command:

dd if=/dev/hda of=/boot/boot.MBR bs=512 count=1

It can then be restored with:

dd if=/boot/boot.MBR of=/dev/hda bs=512 count=1

or if you do not want/need to overwrite the partition table with:

dd if=/boot/boot.MBR of=/dev/hda bs=446 count=1

as the partition table is kept in the last 66 bytes of the MBR.

DISCLAIMER:
Although I have double checked the above, I cannot be held responsible
for any errors. I suggest you try it on a bootable floppy disk before
using it on a hard drive. If it does not work on a floppy disk let me
know.

BTW: I find it easier to boot Linux from a floppy disk or CD-ROM than to
recover an MBR from DOS. The Slackware 3.5 (or greater) CD-ROM
makes a good rescue disk if you have a bios that supports bootable CDs.

types of libraries and their creation

2009-01-05T11:36:00.001-08:00

Why libraries are used:

The following content has been taken from:

http://www.yolinux.com/TUTORIALS/LibraryArchives-StaticAndDynamic.html

This methodology, also known as "shared components" or "archive libraries", groups together multiple compiled object code files into a single file known as a library. Typically C functions/C++ classes and methods which can be shared by more than one application are broken out of the application's source code, compiled and bundled into a library. The C standard libraries and C++ STL are examples of shared components which can be linked with your code. The benefit is that each and every object file need not be stated when linking because the developer can reference the individual library. This simplifies the multiple use and sharing of software components between applications. It also allows application vendors a way to simply release an API to interface with an application. Components which are large can be created for dynamic use, thus the library remain separate from the executable reducing it's size and thus disk space used. The library components are then called by various applications for use when needed.

Linux Library Types:

There are two library types which can be created:

Static libraries (.a): Library of object code which is linked with, and becomes part of the application.
Dynamically linked shared object libraries (.so): There is only one form of this library but it can be used in two ways.
1. Dynamically linked at run time but statically aware. The libraries must be available during compile/link phase. The shared objects are not included into the executable component but are tied to the execution.
2. Dynamically loaded/unloaded and linked during execution (i.e. browser plug-in) using the dynamic linking loader system functions.

Library naming conventions:

Libraries are typically names with the prefix "lib". This is true for all the C standard libraries. When linking, the command line reference to the library will not contain the library prefix or suffix.

Thus the following link command: gcc src-file.c -lm -lpthread
The libraries referenced in this example for inclusion during linking are the math library and the thread library. They are found in /usr/lib/libm.a and /usr/lib/libpthread.a.

Static Libraries: (.a)

How to generate a library:

Compile: cc -Wall -c ctest1.c ctest2.c
Compiler options:
- -Wall: include warnings. See man page for warnings specified.
Create library "libctest.a": ar -cvq libctest.a ctest1.o ctest2.o
List files in library: ar -t libctest.a
Linking with the library:
- cc -o executable-name prog.c libctest.a
- cc -o executable-name prog.c -L/path/to/library-directory -lctest

Example files:

ctest1.c
void ctest1(int *i)
{
*i=5;
}
ctest2.c
void ctest2(int *i)
{
*i=100;
}

prog.c

#include 
void ctest1(int *);
void ctest2(int *);

int main()
{
 int x;
 ctest1(&x);
 printf("Valx=%d\n",x);

 return 0;
}

Historical note: After creating the library it was once necessary to run the command: ranlib ctest.a. This created a symbol table within the archive. Ranlib is now embedded into the "ar" command.

Note for MS/Windows developers: The Linux/Unix ".a" library is conceptually the same as the Visual C++ static ".lib" libraries.

Dynamically Linked "Shared Object" Libraries: (.so)

How to generate a shared object: (Dynamically linked object library file.) Note that this is a two step process.

Create object code
Create library
Optional: create default version using a symbolic link.

Library creation example:

    gcc -Wall -fPIC -c *.c
  gcc -shared -Wl,-soname,libctest.so.1 -o libctest.so.1.0   *.o
  mv libctest.so.1.0 /opt/lib
  ln -sf /opt/lib/libctest.so.1.0 /opt/lib/libctest.so
  ln -sf /opt/lib/libctest.so.1.0 /opt/lib/libctest.so.1

This creates the library libctest.so.1.0 and symbolic links to it.

Compiler options:

-Wall: include warnings. See man page for warnings specified.
-fPIC: Compiler directive to output position independent code, a characteristic required by shared libraries. Also see "-fpic".
-shared: Produce a shared object which can then be linked with other objects to form an executable.
-W1: Pass options to linker.
In this example the options to be passed on to the linker are: "-soname libctest.so.1". The name passed with the "-o" option is passed to gcc.
Option -o: Output of operation. In this case the name of the shared object to be output will be "libctest.so.1.0"

Library Links:

The link to /opt/lib/libctest.so allows the naming convention for the compile flag -lctest to work.
The link to /opt/lib/libctest.so.1 allows the run time binding to work. See dependency below.

Compile main program and link with shared object library:

Compiling for runtime linking with a dynamically linked libctest.so.1.0:

    gcc -Wall -I/path/to/include-files -L/path/to/libraries prog.c -lctest -o prog
Use:
  gcc -Wall -L/opt/lib prog.c -lctest -o prog

Where the name of the library is libctest.so. (This is why you must create the symbolic links or you will get the error "/usr/bin/ld: cannot find -lctest".)
The libraries will NOT be included in the executable but will be dynamically linked during runtime execution.

List Dependencies:

The shared library dependencies of the executable can be listed with the command: ldd name-of-executable

Example: ldd prog

        libctest.so.1 => /opt/lib/libctest.so.1 (0x00002aaaaaaac000)
      libc.so.6 => /lib64/tls/libc.so.6 (0x0000003aa4e00000)
      /lib64/ld-linux-x86-64.so.2 (0x0000003aa4c00000)

Run Program:

Set path: export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH
Run: prog

Man Pages:

gcc - GNU C compiler
ld - The GNU Linker
ldd - List dependencies

Links:

LDP: Shared libraries

Library Path:

In order for an executable to find the required libraries to link with during run time, one must configure the system so that the libraries can be found. Methods available: (Do at least one of the following)

Add library directories to be included during dynamic linking to the file /etc/ld.so.conf

Sample: /etc/ld.so.conf

/usr/X11R6/lib
/usr/lib
...
..
/usr/lib/sane
/usr/lib/mysql
/opt/lib

Add the library path to this file and then execute the command (as root) ldconfig to configure the linker run-time bindings.
You can use the "-f file-name" flag to reference another configuration file if you are developing for different environments.
See man page for command ldconfig.

Add specified directory to library cache: (as root)
ldconfig -n /opt/lib
Where /opt/lib is the directory containing your library libctest.so
(When developing and just adding your current directory: ldconfig -n . Link with -L.)
This will NOT permanently configure the system to include this directory. The information will be lost upon system reboot.

OR

Specify the environment variable LD_LIBRARY_PATH to point to the directory paths containing the shared object library. This will specify to the run time loader that the library paths will be used during execution to resolve dependencies.
(Linux/Solaris: LD_LIBRARY_PATH, SGI: LD_LIBRARYN32_PATH, AIX: LIBPATH, Mac OS X: DYLD_LIBRARY_PATH, HP-UX: SHLIB_PATH)

Example (bash shell): export LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH or add to your ~/.bashrc file:

...
if [ -d /opt/lib ];
then
 LD_LIBRARY_PATH=/opt/lib:$LD_LIBRARY_PATH
fi

...

export LD_LIBRARY_PATH

This instructs the run time loader to look in the path described by the environment variable LD_LIBRARY_PATH, to resolve shared libraries. This will include the path /opt/lib.

Library paths used should conform to the "Linux Standard Base" directory structure.

Library Info:

The command "nm" lists symbols contained in the object file or shared library.

Use the command nm -D libctest.so.1.0
(or nm --dynamic libctest.so.1.0)

0000000000100988 A __bss_start
000000000000068c T ctest1
00000000000006a0 T ctest2
               w __cxa_finalize
00000000001007b0 A _DYNAMIC
0000000000100988 A _edata
0000000000100990 A _end
00000000000006f8 T _fini
0000000000100958 A _GLOBAL_OFFSET_TABLE_
               w __gmon_start__
00000000000005b0 T _init
               w _Jv_RegisterClasses

Man page for nm

Symbol Type	Description
A	The symbol's value is absolute, and will not be changed by further linking.
B	Un-initialized data section
D	Initialized data section
T	Normal code section
U	Undefined symbol used but not defined. Dependency on another library.
W	Doubly defined symbol. If found, allow definition in another library to resolve dependency.

Also see: objdump man page

Library Versions:

Library versions should be specified for shared objects if the function interfaces are expected to change (C++ public/protected class definitions), more or fewer functions are included in the library, the function prototype changes (return data type (int, const int, ...) or argument list changes) or data type changes (object definitions: class data members, inheritance, virtual functions, ...).

The library version can be specified when the shared object library is created. If the library is expected to be updated, then a library version should be specified. This is especially important for shared object libraries which are dynamically linked. This also avoids the Microsoft "DLL hell" problem of conflicting libraries where a system upgrade which changes a standard library breaks an older application expecting an older version of the the shared object function.

Versioning occurs with the GNU C/C++ libraries as well. This often make binaries compiled with one version of the GNU tools incompatible with binaries compiled with other versions unless those versions also reside on the system. Multiple versions of the same library can reside on the same system due to versioning. The version of the library is included in the symbol name so the linker knows which version to link with.

One can look at the symbol version used: nm csub1.o

00000000 T ctest1

No version is specified in object code by default.

ld and object file layout

There is one GNU C/C++ compiler flag that explicitly deals with symbol versioning. Specify the version script to use at compile time with the flag: --version-script=your-version-script-file
Note: This is only useful when creating shared libraries. It is assumed that the programmer knows which libraries to link with when static linking. Runtime linking allows opportunity for library incompatibility.

GNU/Linux, see examples of version scripts here: sysdeps/unix/sysv/linux/Versions

Some symbols may also get version strings from assembler code which appears in glibc headers files. Look at include/libc-symbols.h.

Example: nm /lib/libc.so.6 | more

00000000 A GCC_3.0
00000000 A GLIBC_2.0
00000000 A GLIBC_2.1
00000000 A GLIBC_2.1.1
00000000 A GLIBC_2.1.2
00000000 A GLIBC_2.1.3
00000000 A GLIBC_2.2
00000000 A GLIBC_2.2.1
00000000 A GLIBC_2.2.2
00000000 A GLIBC_2.2.3
00000000 A GLIBC_2.2.4
...
..

Note the use of a version script.

Library referencing a versioned library: nm /lib/libutil-2.2.5.so

..
...
       U strcpy@@GLIBC_2.0
       U strncmp@@GLIBC_2.0
       U strncpy@@GLIBC_2.0
...
..

Links:

Dynamic loading and un-loading of shared libraries using libdl:

These libraries are dynamically loaded / unloaded and linked during execution. Usefull for creating a "plug-in" architecture.

Prototype include file for the library: ctest.h

#ifndef CTEST_H
#define CTEST_H

#ifdef __cplusplus
extern "C" {
#endif

void ctest1(int *);
void ctest2(int *);

#ifdef __cplusplus
}
#endif

#endif

Use the notation extern "C" so the libraries can be used with C and C++. This statement prevents the C++ from name mangling and thus creating "unresolved symbols" when linking.

Load and unload the library libctest.so (created above), dynamically:

#include 
#include 
#include "ctest.h"

int main(int argc, char **argv)
{
 void *lib_handle;
 double (*fn)(int *);
 int x;
 char *error;

 lib_handle = dlopen("/opt/lib/libctest.so", RTLD_LAZY);
 if (!lib_handle)
 {
    fprintf(stderr, "%s\n", dlerror());
    exit(1);
 }

 fn = dlsym(lib_handle, "ctest1");
 if ((error = dlerror()) != NULL)
 {
    fprintf(stderr, "%s\n", error);
    exit(1);
 }

 (*fn)(&x);
 printf("Valx=%d\n",x);

 dlclose(lib_handle);
 return 0;
}

gcc -rdynamic -o progdl progdl.c -ldl

Explanation:

dlopen("/opt/lib/libctest.so", RTLD_LAZY);
Open shared library named "libctest.so".
The second argument indicates the binding. See include file dlfcn.h.
Returns NULL if it fails.
Options:
- RTLD_LAZY: If specified, Linux is not concerned about unresolved symbols until they are referenced.
- RTLD_NOW: All unresolved symbols resolved when dlopen() is called.
- RTLD_GLOBAL: Make symbol libraries visible.
dlsym(lib_handle, "ctest1");
Returns address to the function which has been loaded with the shared library..
Returns NULL if it fails.
Note: When using C++ functions, first use nm to find the "mangled" symbol name or use the extern "C" construct to avoid name mangling.
i.e. extern "C" void function-name();

Object code location: Object code archive libraries can be located with either the executable or the loadable library. Object code routines used by both should not be duplicated in each. This is especially true for code which use static variables such as singleton classes. A static variable is global and thus can only be represented once. Including it twice will provide unexpected results. The programmer can specify that specific object code be linked with the executable by using linker commands which are passed on by the compiler.

Use the "-Wl" gcc/g++ compiler flag to pass command line arguments on to the GNU "ld" linker.

Example makefile statement: g++ -rdynamic -o appexe $(OBJ) $(LINKFLAGS) -Wl,--whole-archive -L{AA_libs} -laa -Wl,--no-whole-archive $(LIBS)

--whole-archive: This linker directive specifies that the libraries listed following this directive (in this case AA_libs) shall be included in the resulting output even though there may not be any calls requiring its pressence. This option is used to specify libraries which the loadable libraries will require at run time.
-no-whole-archive: This needs to be specified whether you list additional object files or not. The gcc/g++ compiler will add its own list of archive libraries and you would not want all the object code in the archive library linked in if not needed. It toggles the behavior back to normal for the rest of the archive libraries.

Man pages:

dlopen() - gain access to an executable object file
dclose() - close a dlopen object
dlsym() - obtain the address of a symbol from a dlopen object
dlvsym() - Programming interface to dynamic linking loader.
dlerror() - get diagnostic information

Links:

Shared Libraries-Dynamic Loading and Unloading
GNOME Glib dynamic loading of modules - cross platform API for dynamically loading "plug-ins".

C++ class objects and dynamic loading:

C++ and name mangling:

When running the above "C" examples with the "C++" compiler one will quickly find that "C++" function names get mangled and thus will not work unless the function definitions are protected with extern "C"{}.

Note that the following are not equivalent:

extern "C"
{
 int functionx();
}

extern "C" int functionx();

The following are equivalent:

extern "C"
{
 extern int functionx();
}

extern "C" int functionx();

Dynamic loading of C++ classes:

The dynamic library loading routines enable the programmer to load "C" functions. In C++ we would like to load class member functions. In fact the entire class may be in the library and we may want to load and have access to the entire object and all of its member functions. Do this by passing a "C" class factory function which instantiates the class.

The class ".h" file:

class Abc {

...
...

};

// Class factory "C" functions

typedef Abc* create_t;
typedef void destroy_t(Abc*);

The class ".cpp" file:

Abc::Abc()
{
  ...
}

extern "C"
{
 // These two "C" functions manage the creation and destruction of the class Abc

 Abc* create()
 {
    return new Abc;
 }

 void destroy(Abc* p)
 {
    delete p;   // Can use a base class or derived class pointer here
 }
}

This file is the source to the library. The "C" functions to instantiate (create) and destroy a class defined in the dynamically loaded library where "Abc" is the C++ class.

Main executable which calls the loadable libraries:

// load the symbols
  create_t* create_abc = (create_t*) dlsym(lib_handle, "create");

...
...

  destroy_t* destroy_abc = (destroy_t*) dlsym(lib_handle, "destroy");

...
...

Pitfalls:

The new/delete of the C++ class should both be provided by the executable or the library but not split. This is so that there is no surprise if one overloads new/delete in one or the other.

Links:

Comparison to the Microsoft DLL:

The Microsoft Windows equivalent to the Linux / Unix shared object (".so") is the ".dll". The Microsoft Windows DLL file usually has the extension ".dll", but may also use the extension ".ocx". On the old 16 bit windows, the dynamically linked libraries were also named with the ".exe" suffix. "Executing" the DLL will load it into memory.

The Visual C++ .NET IDE wizard will create a DLL framework through the GUI, and generates a ".def" file. This "module definition file" lists the functions to be exported. When exporting C++ functions, the C++ mangled names are used. Using the Visual C++ compiler to generate a ".map" file will allow you to discover the C++ mangled name to use in the ".def" file. The "SECTIONS" label in the ".def" file will define the portions which are "shared". Unfortunately the generation of DLLs are tightly coupled to the Microsoft IDE, so much so that I would not recomend trying to create one without it.

The Microsoft Windows C++ equivalent functions to libdl are the following functions:

::LoadLibrary() - dlopen()
::GetProcAddress() - dlsym()
::FreeLibrary() - dlclose()

[Potential Pitfall]: Microsoft Visual C++ .NET compilers do not allow the linking controll that the GNU linker "ld" allows (i.e. --whole-archive, -no-whole-archive). All symbols need to be resolved by the VC++ compiler for both the loadable library and the application executable individually and thus it can cause duplication of libraries when the library is loaded. This is especially bad when using static variables (i.e. used in singleton patterns) as you will get two memory locations for the static variable, one used by the loadable library and the other used by the program executable. This breaks the whole static variable concept and the singleton pattern. Thus you can not use a static variable which is referenced by by both the loadable library and the application executable as they will be unique and different. To use a unique static variable, you must pass a pointer to that static variable to the other module so that each module (main executable and DLL library) can use the same instatiation. On MS/Windows you can use shared memory or a memory mapped file so that the main executable and DLL library can share a pointer to an address they both will use.

Cross platform (Linux and MS/Windows) C++ code snippet:

Include file declaration: (.h or .hpp)

class Abc{
public:
 static Abc* Instance(); // Function declaration. Could also be used as a public class member function.

private:
 static Abc *mInstance;      // Singleton. Use this declaration in C++ class member variable declaration.
 ...
}

C/C++ Function source: (.cpp)

/// Singleton instantiation
Abc* Abc::mInstance = 0;   // Use this declaration for C++ class member variable
                         // (Defined outside of class definition in ".cpp" file)

// Return unique pointer to instance of Abc or create it if it does not exist.
// (Unique to both exe and dll)

static Abc* Abc::Instance() // Singleton
{
#ifdef WIN32
  // If pointer to instance of Abc exists (true) then return instance pointer else look for
  // instance pointer in memory mapped pointer. If the instance pointer does not exist in
  // memory mapped pointer, return a newly created pointer to an instance of Abc.

  return mInstance ?
     mInstance : (mInstance = (Abc*) MemoryMappedPointers::getPointer("Abc")) ?
     mInstance : (mInstance = (Abc*) MemoryMappedPointers::createEntry("Abc",(void*)new Abc));
#else
  // If pointer to instance of Abc exists (true) then return instance pointer
  // else return a newly created pointer to an instance of Abc.

  return mInstance ? mInstance : (mInstance = new Abc);
#endif
}

Windows linker will pull two instances of object, one in exe and one in loadable module. Specify one for both to use by using memory mapped pointer so both exe and loadable library point to same variable or object.
Note that the GNU linker does not have this problem.

For more on singletons see the YoLinux.com C++ singleton software design pattern tutorial.

Cross platform programming of loadable libraries:

#ifndef USE_PRECOMPILED_HEADERS
#ifdef WIN32
#include 
#include 
#else
#include 
#include 
#endif
#include 
#endif

  using namespace std;

#ifdef WIN32
  HINSTANCE lib_handle;
#else
  void *lib_handle;
#endif

  // Where retType is the pointer to a return type of the function
  // This return type can be int, float, double, etc or a struct or class.

  typedef retType* func_t;

  // load the library -------------------------------------------------
#ifdef WIN32
  string nameOfLibToLoad("C:\opt\lib\libctest.dll");
  lib_handle = LoadLibrary(TEXT(nameOfLibToLoad.c_str()));
  if (!lib_handle) {
      cerr << "Cannot load library: " << lib_handle =" dlopen(nameOfLibToLoad.c_str()," fn_handle =" (func_t*)" fn_handle=" (func_t*)" dlsym_error =" dlerror();">

finally a break

2008-12-19T23:19:00.000-08:00

finally a break until i resume for my doctorate at Georgia Institute of Technology at Center for Experimental Research in Computing Systems under Prof. Karsten Schwan.........but u guys can let me know if something interesting comes up relating systems, ill be happy to put it here :) :) :)

PCI passthrough for Xen

2008-12-16T09:56:00.000-08:00

2.2. How do I hide a pci device from dom0?

In Xen 2.x, you can add the physdev_dom0_hide parameter (see XenBooting for more parameters) to hide one or more pci devices to Dom0, so you can affect them to domU.

Pci slots MUST be formatted like this:

(nn:nn.n)

(03:06.1) is correct
(03:6.1) is NOT correct

You can get the pci bus address on a Linux system by using the lspci command (only the last bits are relevant):

lspci | grep Ethernet
0000:02:03.0 Ethernet controller: Intel Corp. 82546EB Gigabit Ethernet Controller (Copper) (rev 01)
0000:02:03.1 Ethernet controller: Intel Corp. 82546EB Gigabit Ethernet Controller (Copper) (rev 01)

Then:

to hide the second network interface from dom0, you can then append physdev_dom0_hide=(02:03.1) to your kernel /boot/xen-2.0.gz.
to hide multiple pci devices, simply concatenate all the pci slots address like this: physdev_dom0_hide=(02:03.0)(02:03.1)

If everything went ok, you should see the following lines after a reboot:

(XEN) Hiding PCI device 02:03.0 from DOM0
(XEN) Hiding PCI device 02:03.1 from DOM0

Also refer to the link:
http://wiki.xensource.com/xenwiki/XenFaq#head-d5a7a247a5168517291228a6f02fd74b419badeb

Windows Installation using USB key device

2008-12-09T18:22:00.000-08:00

Installing Windows XP from USB

I wrote this piece back in March 2006 before transferring it to my new blog. It’s quite possible that bits and pieces of this HOWTO are outdated. If any are and you find a solution, then please leave a comment for everyone to peruse.

What follows is a recipe on how to install Windows XP from USB. You can cut right to the case or read about how it happened to me, first. Sit tight, you’re in for a ride!

Case In Point

You’ve got this Acer laptop with a malfunctioning DVD drive. Specifically, you removed it from its bay and now Windows XP crashes when you plug it back in. Knowledge Base article 310664 matches it to a tee.

Provided solution: update to Service Pack 1, or disable DMA in the BIOS. Current situation: running SP2 and blessed with a “user-friendly” BIOS that doesn’t allow you to disable DMA.

Your luck just ran out, but - being the geek that you are - you don’t whine about it. Instead, you come up with this bright idea: let’s install Linux, that’ll work some magic! And so you get Gentoo Linux installed using a home-brewn LiveUSB distribution. And much to your dismay, proper hotplugging implementation for your laptop is lacking. Curses.

So finally, you call the support line to have them collect your laptop for repair. One problem here: you’re betting they won’t like seeing Linux boot up and, hoping to forego additional service costs, it’s up to you to reinstall Windows XP.

One problem here: you don’t have a working CD-ROM drive to install Windows XP off, and from experience you know that Windows XP won’t install from an external drive. Blasted!

Enter the cavalry.

Disclaimer

The procedure here is provided without any warranty or support whatsoever. It worked for me and so it might for you. It also might blast you and your laptop to pieces. Some of the procedures described here can wipe all of your data. Thought I’d warn you.

I don’t mean to be inventive here, either. In fact, all the technology has been laid out for us and is readily available. All I’m doing is wrapping it all up in writing. Shouts out to everyone who provided the technology!

That being said, let’s get down to business.

Ground Zero: Ingredients

You’ll be needing ample space on a USB storage device like a thumbdrive or external disk drive. 1 GB of storage will suffice. Possibly, you could access the setup files from a network share in which case you’ll only need a 256 MB thumbdrive, but I haven’t tried that.

Personally, I booted off a 512 MB thumbdrive and put the Windows XP setup files on an external disk drive. So a combination will do just fine, but just make sure that you’ve got sufficient USB ports: all USB devices will need to be plugged in and stay plugged in while we’re booted into BartPE later on!

You’ll also need a bunch of software, including a valid copy of Windows XP. The other software components can be downloaded freely:

Q: Why Windows 2003 Server Service Pack 1? I want to install XP!

A: Well that’s exactly the trick, though it’s a bit technical. During booting, Windows resets the USB port and loses your boot device. That’s why an installation from USB normally fails. But by using Windows 2003 Server’s ramdisk feature in the early phases of booting, we circumvent that problem.

Don’t be mislead though: you’ll be installing Windows XP all the same. It’s just that we’ll be using two Windows 2003 Server files from the Service Pack to do it.

You can also perform a Windows 2003 Server installation using this exact same procedure, save that you’ll be needing to update to Windows 2003 Server Service Pack 1 instead of Windows XP SP2, below. That’s assuming, of course, that you’ve got a valid copy of Windows 2003 Server.

You will also need an already functional Windows system to prep the USB disk from. In addition, your target system (the one that needs reinstalling) should support booting from USB. Virtually all modern ones do though.

Finally, not all cheaper USB devices properly support being booted from. The BartPE forums have a partial list of ones that don’t.

Step One: Preparing the Windows XP Setup Files

For this procedure to work correctly, you’ll need Windows XP to be at Service Pack 1 or Service Pack 2. Many newer Windows XP CDs already have either one included.

You can check by inserting the CD-ROM and looking at the files in the top folder. There should be a file named WIN51P.SP1 or WIN51P.SP2. If you see either one, fast forward to step two.If there is no such file, or you’ve got a SP1 CD that you would like to update to SP2, proceed as follows:

Copy the Windows XP CD-ROM to a location on your hard drive. We’ll be assuming C:\winxpcd. I strongly advise you to use such a vanilla CD. Users have reported strange happenings with custom XP distributions.
Download the Windows XP Service Pack 2 Network Installation Package.
Unpack Windows XP SP2 by running WindowsXP-KB835935-SP2-ENU.exe -x. Note the -x parameter that will allow you to specify a location to unpack to. We’ll be assuming C:\winxpsp2.
Integrate SP2 into the setup files by running C:\winxpsp2\i386\update\update.exe /integrate:c:\winxpcd.

If your system needs special drivers to operate, this would be the time to include them (called “slipstreaming”). Doing so isn’t exactly trivial and should only be necessary if you, say, require non-standard SATA support. The process is beyond the scope of this article, but fortunately Google is your friend.

Step Two: Prepping BartPE

First we’ll prep the USB drive we’ll be using for booting. This is by far the largest step in the process, though it shouldn’t be difficult for anyone to follow. It requires a functional Windows system.

Install BartPE. We’ll be assuming that you install to C:\pebuilder.
Unpack PE2USB to C:\pebuilder.
Unpack Windows 2003 SP1 by running WindowsServer2003-KB889101-SP1-x86-ENU.exe -x (that’s assuming that you downloaded the English version). Note the -x parameter that will allow you to specify a location to unpack to. We’ll be assuming C:\server2003sp1.
Create a folder named C:\pebuilder\srsp1.
Copy C:\server2003sp1\i386\setupldr.bin to C:\pebuilder\srsp1.
Expand and copy ramdisk.sys by running expand -r C:\server2003sp1\i386\ramdisk.sy_ C:\pebuilder\srsp1.
Start PE Builder from the Start Menu. Point “Source:” to the location of your Windows XP setup files containing SP1 or SP2.
Make sure the “Output:” is set to “BartPE”. If using SP2: go into the Plugins configuration dialog and enable “RpcSS needs to launch DComLaunch Service First - SP2 only”.
Hit “Build”.
When it’s done building, insert the USB drive you intend to boot from and run C:\pebuilder\pe2usb -f [drive:], replacing [drive:] with the location of your USB drive. This will destroy any data on your USB drive! The drive shouldn’t be partitioned any larger than 1 GB. If it is, then repartition it using Computer Management in the Control Panel.
Copy the Windows XP setup files to the USB drive (or another USB device if you have insufficient space left). You’ll only need the i386 folder.

If no errors occured in the process, you should now be all set to boot from USB!

Note: it’s possible to use Windows 2003 Server as the operating system to boot from, but I don’t recommend it. Many of BartPE’s included utilities won’t work properly.

Step Three: Configuring the BIOS

You should now reboot and go into the BIOS configuration to boot from USB. Instructions for doing so wildly from system to system, but generally entail the following:

Reboot the system.
While booting (before Windows starts loading), get into the BIOS configuration screen by hitting something like F1, F2, Delete or Escape. Hotkey instructions are generally provided on the screen.
Go to the section that contains your boot devices.
With your USB drive plugged in, the USB drive should be listed. If it isn’t, your system might not support booting from USB. Assuming that it is supported (as is the case with virtually all modern hardware), promote your USB drive to the primary boot device.
Exit from the BIOS configuration, saving all changes.

If you’re completely new to BIOS configuration, BIOS for Beginners over at Tom’s Hardware might be a good primer. Be aware though, that you can seriously screw up your system by providing incorrect settings!

Step Four: Booting into BartPE

Assuming that you properly configured your BIOS and your USB device supports booting, BartPE should now load. Depending on the speed of your USB device, this may take a while.

If it isn’t working, then double-check the following before making a scene:

Is your BIOS properly configured for booting from the USB disk? (Is the USB device listed and does it have top priority?)
Have you correctly prepared the USB disk in step two? (Restart the procedure.)
Does your USB stick properly support being booted from? (Try another one!)

Step Five: Prepping the Hard Disk

You need to make sure that your hard drive is partitioned and formatted properly. Especially if you’ve had Linux or some other operating system on it, you’ll need to repartition and format it. BartPE contains DiskPart for disk partitioning and A43 File Manager to format your drive.

If you are sure that your hard drive is set up properly (i.e. it has only run Windows, it contains a valid FAT or NTFS partition) then you can safe yourself the hassle and skip this step.

This procedure will destroy any data on the hard drive.

To repartition:

From the Go menu, navigate to DiskPart.
Enter the commands needed to repartition your drive. For example, try the following: select disk 0 (select the first disk), clean (purges the entire drive, essentially resetting it), create partition primary (creates a single partition from the entire disk), assign (assign the partition a drive letter), exit (quits DiskPart).

This procedure will destroy any data on the hard drive.

To format:

From the Go menu, navigate to the A43 File Manager.
Right-click on the target drive (e.g. C:) and click Format. Should be self-explanatory.

Step Six: Launching Windows XP Setup

With your drive all ready, you can now launch the Windows XP setup with a few custom parameters. Let’s assume that the files are available at D:\i386.

Plugging in a device now won’t work. Remember that all USB devices will need to be plugged in right from the start while using BartPE.

Run D:\i386\winnt32.exe /syspart:C: /tempdrive:C: /makelocalsource. Replace C: with the drive you want to install Windows to.
Proceed with the installation. When asked to convert the installation volume to NTFS, answer No. The setup program incorrectly believes that your USB drive (which is formatted as FAT) needs conversion.

The setup program will then silently close, which might make you think that something went wrong. Don’t worry though. Reboot your system, remove the USB drive(s) and – if all went well – you can now finish setting up Windows XP. Congratulations!

All comments are highly welcome. Hope to have helped some of you out!

Xen PCI passthrough

2008-12-05T13:06:00.000-08:00

Edit PageHistory Diff Info LikePages

Xen has a feature which lets you pass a PCI device through to an unpriviledged domain. Only the unpriviledged domain has access to this specific PCI device – the priviledged domain binds to the PCI device with a dummy drive that stops the dom0 from accessing it as well.

This could be used for passing a SCSI controller or NIC through to one domain, eg. for a file server or FireWall domU.

What you'll need

You'll have to compile in the right PCI backend driver for Xen.

    CONFIG_XEN_PCIDEV_FRONTEND=y (for DomU kernels)
   CONFIG_XEN_PCIDEV_BACKEND=y (for Dom0 kernels)

You'll also need the PCI ID of the device in question, in the xx:xx:x notation displayed by lspci(8). Note that the formatting is important – don't drop or add leading 0's!

In some cases it is required to set the pciback.permissive option – this is for drivers which need to write to registers on the PCI device itself. Apparently some NICs will fail to operate without this, and from what I've read it doesn't hurt to have it enabled regardless.

Kernel options

In grub.conf, add something like the following to your module line corresponding to the dom0 kernel.

pciback.permissive pciback.hide=(03:00.0)

This assumes the device we are passing through has a PCI ID of 03:00.0.

Xen config file

In your Xen config file for the domain, add a line that looks like the following, again assuming a PCI ID of 03:00.0:

pci = ['03:00.0']

Note that if this is a network interface, it will come up in addition to any you specify with your vif config option.

Xen DomU creation

2008-12-05T12:57:00.000-08:00

Creating a Virtual Server - domU

There are 3 options of what to run DomU on:

File Based Image
LVM Based
Physical Partition

1. A file based image is the quickest to setup, however has poor/terrible IO performance. The virtual server is limited to the initial size of the image created also. The file based Image can however be easily mounted in a rescue system, and easily backed-up.

2. LVM for domU is the industry standard. After the initial setup of LVM, as described here, it is a dream to manage. LVM partitions can be resized afterwards!!! Due to this "resizing" capability and flexibility, its use for Xen Virtual Servers is ideal. They also have much better IO performance than file-based. I dont know about mounting these partitions however in a rescue system. Something to try out... -).

3. Physical Partitions have the best IO, but are difficult to alter and inflexible.

[edit]

File Based Setup of Virtual Disk

mkdir -p /xens/burkesys/
dd if=/dev/zero of=/xens/burkesys/diskimage.img bs=1024k count=5000
dd if=/dev/zero of=/xens/burkesys/swapimage.img bs=1024k count=512

mkfs.ext3 /xens/burkesys/diskimage.img
mkswap /xens/burkesys/swapimage.img

mount -o loop /xens/burkesys/diskimage.img /mnt

[edit]

LVM Based Setup of Virtual Disk

lvcreate -n twister --size 2g main-vol2
lvcreate -n twisterswp --size 512m main-vol2

mkfs.ext3 /dev/main-vol2/twister
mkswap /dev/main-vol2/twisterswp

mount -o loop /dev/main-vol2/twister /mnt

[edit]

Debootstrap New OS onto Virtual Disk

debootstrap --arch i386 sarge /mnt http://ftp.de.debian.org/debian/

mv /mnt/lib/tls /mnt/lib/tls.disabled

cp /etc/apt/sources.list /mnt/etc/apt/
vi /mnt/etc/apt/sources.list

cp -a /lib/modules/2.6.16-1-xen-k7/ /mnt/lib/modules/

cp /etc/resolve.conf /mnt/etc/

cp /etc/network/interfaces /mnt/etc/network/
vi /mnt/etc/network/interfaces

#To use Specific IP address - edit the /mnt/etc/network/interfaces manually.
#To use DHCP, edit and include the following:
   # The loopback network interface
  auto lo
  iface lo inet loopback
  # The primary network interface
  auto eth0
  iface eth0 inet dhcp

vi /mnt/etc/hostname

vi /mnt/etc/fstab
proc            /proc       proc    defaults    0 0
/dev/sda1       /           ext3    defaults,errors=remount-ro    0 1
/dev/sda2       none        swap    sw          0 0

[edit]

Setup domU Xen Config

umount /mnt

vi /etc/xen/burkesys
kernel = "/boot/vmlinuz-2.6.16-1-xen-k7"
ramdisk = "/boot/initrd.img-2.6.16-1-xen-k7"
memory = 192
name = "burkesys"
vif = ['bridge=xenbr0']
#File Based:# disk = ['file:/xens/burkesys/diskimage.img,sda1,w','file:/xens/burkesys/swapimage.img,sda2,w']
#LVM Based:# disk = ['phy:/dev/main-vol2/twister,sda1,w', 'phy:/dev/main-vol2/twister,sda2,w']
#DHCP - remove the ip, gateway and netmask lines, and include:# dhcp = "dhcp"
ip = "ip add"
gateway = "ip add"
netmask = "255.255.0.0"
root = "/dev/sda1 ro"
extra = '4'

ln -s /etc/xen/burkesys /etc/xen/auto/burkesys
//link in the config file so that the Virtal Machine starts on Bootup of Dom0

[edit]

Run and Update DomU

xm create burkesys -c
// CTRL + ]  Gets out of the Console

All going well you should see domU booting up. Login as "root" with no password. Execute the following commands to update:

passwd
pwconv         #Very important. See explaination below.

apt-get update
apt-get upgrade

apt-get install module-init-tools   //for iptables
apt-get install iptables

iptables -L

[edit]

Fix for passwd and shadow with Debootstrap install

By default, when a system is installed using debootstrap, it does *not* create /etc/shadow, and simply creates /etc/passwd with the root password hashed. Looking at /etc/passwd after a debootstrap'ed system is running, you will see the root user and the root's password hashed. If there were other users installed on the same system, they could read the root's password hash, because /etc/passwd is world readable.

A more secure method is to store only the usernames in /etc/passwd and to store the password hash in /etc/shadow which has special permissions. Therefore any user won't be able to access the password hashes for other users. To convert passwd to shadow, its really very simple, just run:

pwconv

Take a look at /etc/passwd before and after, and also at /etc/shadow. Thanks to davisc for pointing out this oversight to me :)

[edit]

Bug Fix for XEN3 and File Based Images

When the base machine dom0 is rebooted - the xen machines (xm create vm01 -c) wont work and will complain. A manual fix is to mount -o loop /xens/image.img /xens/mnt and then umount.

However if you add this line into /etc/init.d/xend

modprobe loop max_loop=64

before echo -n "Starting $DESC: " It will work automatically. Note - this seems to be only for file based storage :-\

URLS at: http://lists.alioth.debian.org/pipermail/pkg-xen-devel/2006-June/000563.html

http://lists.xensource.com/archives/html/xen-users/2006-04/msg00968.html

Grub installation

2008-12-05T12:45:00.000-08:00

3. Installation

In order to install GRUB as your boot loader, you need to first install the GRUB system and utilities under your UNIX-like operating system (see section A. How to obtain and build GRUB). You can do this either from the source tarball, or as a package for your OS.

After you have done that, you need to install the boot loader on a drive (floppy or hard disk). There are two ways of doing that - either using the utility grub-install (see section 16. Invoking grub-install) on a UNIX-like OS, or by running GRUB itself from a floppy. These are quite similar, however the utility might probe a wrong BIOS drive, so you should be careful.

Also, if you install GRUB on a UNIX-like OS, please make sure that you have an emergency boot disk ready, so that you can rescue your computer if, by any chance, your hard drive becomes unusable (unbootable).

GRUB comes with boot images, which are normally put in the directory `/usr/share/grub/i386-pc'. If you do not use grub-install, then you need to copy the files `stage1', `stage2', and `*stage1_5' to the directory `/boot/grub'. Hereafter, the directory where GRUB images are initially placed (normally `/usr/share/grub/i386-pc') will be called the image directory, and the directory where the boot loader needs to find them (usually `/boot/grub') will be called the boot directory.

3.1 Creating a GRUB boot floppy
3.2 Installing GRUB natively
3.3 Installing GRUB using grub-install
3.4 Making a GRUB bootable CD-ROM

[ < ]

[ > ]

[ << ]

[ Up ]

[ >> ]

[Top]

[Contents]

[Index]

[ ? ]

3.1 Creating a GRUB boot floppy

To create a GRUB boot floppy, you need to take the files `stage1' and `stage2' from the image directory, and write them to the first and the second block of the floppy disk, respectively.

Caution: This procedure will destroy any data currently stored on the floppy.

On a UNIX-like operating system, that is done with the following commands:

# cd /usr/share/grub/i386-pc
# dd if=stage1 of=/dev/fd0 bs=512 count=1
1+0 records in
1+0 records out
# dd if=stage2 of=/dev/fd0 bs=512 seek=1
153+1 records in
153+1 records out
#

The device file name may be different. Consult the manual for your OS.

[ < ]

[ > ]

[ << ]

[ Up ]

[ >> ]

[Top]

[Contents]

[Index]

[ ? ]

3.2 Installing GRUB natively

Caution: Installing GRUB's stage1 in this manner will erase the normal boot-sector used by an OS.

GRUB can currently boot GNU Mach, Linux, FreeBSD, NetBSD, and OpenBSD directly, so using it on a boot sector (the first sector of a partition) should be okay. But generally, it would be a good idea to back up the first sector of the partition on which you are installing GRUB's stage1. This isn't as important if you are installing GRUB on the first sector of a hard disk, since it's easy to reinitialize it (e.g. by running `FDISK /MBR' from DOS).

If you decide to install GRUB in the native environment, which is definitely desirable, you'll need to create a GRUB boot disk, and reboot your computer with it. Otherwise, see 3.3 Installing GRUB using grub-install.

Once started, GRUB will show the command-line interface (see section 12.1 The flexible command-line interface). First, set the GRUB's root device(4) to the partition containing the boot directory, like this:

grub> root (hd0,0)

If you are not sure which partition actually holds this directory, use the command find (see section 13.3.11 find), like this:

grub> find /boot/grub/stage1

This will search for the file name `/boot/grub/stage1' and show the devices which contain the file.

Once you've set the root device correctly, run the command setup (see section 13.3.34 setup):

grub> setup (hd0)

This command will install the GRUB boot loader on the Master Boot Record (MBR) of the first drive. If you want to put GRUB into the boot sector of a partition instead of putting it in the MBR, specify the partition into which you want to install GRUB:

grub> setup (hd0,0)

If you install GRUB into a partition or a drive other than the first one, you must chain-load GRUB from another boot loader. Refer to the manual for the boot loader to know how to chain-load GRUB.

After using the setup command, you will boot into GRUB without the GRUB floppy. See the chapter 4. Booting to find out how to boot your operating systems from GRUB.

[ < ]

[ > ]

[ << ]

[ Up ]

[ >> ]

[Top]

[Contents]

[Index]

[ ? ]

3.3 Installing GRUB using grub-install

Caution: This procedure is definitely less safe, because there are several ways in which your computer can become unbootable. For example, most operating systems don't tell GRUB how to map BIOS drives to OS devices correctly--GRUB merely guesses the mapping. This will succeed in most cases, but not always. Therefore, GRUB provides you with a map file called the device map, which you must fix if it is wrong. See section 15.3 The map between BIOS drives and OS devices, for more details.

If you still do want to install GRUB under a UNIX-like OS (such as GNU), invoke the program grub-install (see section 16. Invoking grub-install) as the superuser (root).

The usage is basically very simple. You only need to specify one argument to the program, namely, where to install the boot loader. The argument can be either a device file (like `/dev/hda') or a partition specified in GRUB's notation. For example, under Linux the following will install GRUB into the MBR of the first IDE disk:

# grub-install /dev/hda

Likewise, under GNU/Hurd, this has the same effect:

# grub-install /dev/hd0

If it is the first BIOS drive, this is the same as well:

# grub-install '(hd0)'

Or you can omit the parentheses:

# grub-install hd0

But all the above examples assume that GRUB should use images under the root directory. If you want GRUB to use images under a directory other than the root directory, you need to specify the option `--root-directory'. The typical usage is that you create a GRUB boot floppy with a filesystem. Here is an example:

# mke2fs /dev/fd0
# mount -t ext2 /dev/fd0 /mnt
# grub-install --root-directory=/mnt fd0
# umount /mnt

Another example is when you have a separate boot partition which is mounted at `/boot'. Since GRUB is a boot loader, it doesn't know anything about mountpoints at all. Thus, you need to run grub-install like this:

# grub-install --root-directory=/boot /dev/hda

By the way, as noted above, it is quite difficult to guess BIOS drives correctly under a UNIX-like OS. Thus, grub-install will prompt you to check if it could really guess the correct mappings, after the installation. The format is defined in 15.3 The map between BIOS drives and OS devices. Please be quite careful. If the output is wrong, it is unlikely that your computer will be able to boot with no problem.

Note that grub-install is actually just a shell script and the real task is done by the grub shell grub (see section 15. Invoking the grub shell). Therefore, you may run grub directly to install GRUB, without using grub-install. Don't do that, however, unless you are very familiar with the internals of GRUB. Installing a boot loader on a running OS may be extremely dangerous.

[ < ]

[ > ]

[ << ]

[ Up ]

[ >> ]

[Top]

[Contents]

[Index]

[ ? ]

3.4 Making a GRUB bootable CD-ROM

GRUB supports the no emulation mode in the El Torito specification(5). This means that you can use the whole CD-ROM from GRUB and you don't have to make a floppy or hard disk image file, which can cause compatibility problems.

For booting from a CD-ROM, GRUB uses a special Stage 2 called `stage2_eltorito'. The only GRUB files you need to have in your bootable CD-ROM are this `stage2_eltorito' and optionally a config file `menu.lst'. You don't need to use `stage1' or `stage2', because El Torito is quite different from the standard boot process.

Here is an example of procedures to make a bootable CD-ROM image. First, make a top directory for the bootable image, say, `iso':

$ mkdir iso

Make a directory for GRUB:

$ mkdir -p iso/boot/grub

Copy the file `stage2_eltorito':

$ cp /usr/share/grub/i386-pc/stage2_eltorito iso/boot/grub

If desired, make the config file `menu.lst' under `iso/boot/grub' (see section 5. Configuration), and copy any files and directories for the disc to the directory `iso/'.

Finally, make a ISO9660 image file like this:

$ mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot \
   -boot-load-size 4 -boot-info-table -o grub.iso iso

This produces a file named `grub.iso', which then can be burned into a CD (or a DVD). mkisofs has already set up the disc to boot from the boot/grub/stage2_eltorito file, so there is no need to setup GRUB on the disc. (Note that the -boot-load-size 4 bit is required for compatibility with the BIOS on many older machines.)

You can use the device `(cd)' to access a CD-ROM in your config file. This is not required; GRUB automatically sets the root device to `(cd)' when booted from a CD-ROM. It is only necessary to refer to `(cd)' if you want to access other drives as well.

Also dont forget to change the root partition that you get during find of stage 2 in the grub.conf or menu.lst file so that your partitions are accessible while booting.

xen networking configuration

2008-11-09T09:59:00.000-08:00

http://wiki.xensource.com/xenwiki/XenNetworking

iptables -A FORWARD -m physdev --physdev-in eth0
--physdev-out '!' eth0  -j ACCEPT

iptables -A FORWARD -m physdev --physdev-out eth0
--physdev-in '!' eth0  -j ACCEPT

you can change the bridge name from xenbr0 using:

(network-script 'network-bridge bridge=mybridge')

you can change the bridge name from xenbr0 using:
```
(network-script 'network-bridge bridge=mybridge')
```
in xend-config.sxp and rebooting or restarting xend
remember to configure the bridge to attach to in the domU's config file using:
```
vif=[ 'bridge=mybridge' ]
```
or perhaps something like:
```
vif=[ 'mac=00:16:3e:01:01:01,bridge=mybridge' ]
```
you can create multiple network interfaces, and attach them to different bridges using:
```
vif=[ 'mac=00:16:3e:70:01:01,bridge=br0', 'mac=00:16:3e:70:02:01,bridge=br1' ]
```

if you want to use multiple bridges, you must create them yourself, either manually, or via your own startup script, or via a custom script to replace network-bridge. For example:

$ cd /etc/xen/scripts
$ cp network-bridge network-custom
$ cp vif-bridge vif-custom
$ vi /etc/xen/xend-config.sxp
        (network-script network-custom)
        (vif-script vif-custom)
$ vi network-custom
        # whatever you want

before you connect a physical interface to a bridge, remember to reset it's mac and turn arp off. For example:

# ip link set eth1 down
# ip link set eth1 mac fe:ff:ff:ff:ff:ff arp off
# brctl addif br1 eth1
# ip link set eth1 up

With Xen 3.0 the best method for additional bridges is to use the default Xen scripts with a slight modification. Following the XenBug #332. For example in a two bridge network with eth0 and eth1. Create /etc/xen/scripts/my-network-script with
```
#!/bin/sh
dir=$(dirname "$0")
"$dir/network-bridge" "$@" vifnum=0
"$dir/network-bridge" "$@" vifnum=1
```
With Xen 3.2.1 (tested on Debian Etch 4.0r3), here is a script example that creates two virtual interfaces corresponding to the 2 physical network interfaces
```
# xemacs /etc/xen/scripts/network-bridge-wrapper
#!/bin/sh
/etc/xen/scripts/network-bridge $1 netdev=eth0
/etc/xen/scripts/network-bridge $1 netdev=eth1
```

The $1 will use the argument of xend (in the /etc/xen/xend-config.sxp configuration file). If there is a default physical network interface, the standard network-bridge script of Xen will create a vif for this interface, and not the other ones also.

(Additional note by steve_from_moreover - May be stating the obvious but remember to do - chmod 755 /etc/xen/scripts/my-network-script or when you reboot it will silently not be able to run this script).
On SuSE Linux (at least), each interface requires an ifcfg script in /etc/sysconfig/network, e.g. /etc/sysconfig/network/ifcfg-eth1. Otherwise, network-bridge will create the bridge with no interfaces attached.
Then change /etc/xen/xend-config.sxp with the following (network-script my-network-script).
The same principle can apply to networks without a physical ethernet device. Use a dummy interface with
```
"$dir/network-bridge" "$@" vifnum=2 netdev=dummy0
```