Cross-compiling (ES series)


From SamyGO
Jump to: navigation, search

This is Work In Progress !!

No answers/solutions can be expected here yet. If you need quick answers Google the forums.

@developers:
Do not change or update this page without first checking/asking on the "Talk" page or support forum thread for latest status/info.


This is an introduction to cross-compiling for MST10P (MStar/MediaTek) based TV sets. It is primary intended as a crash course for getting even a novice to be able to quickly compile his/her own programs to run on their TV sets. As such, we will assume that you are using a Windows based PC with a basic installation of Cygwin. The modification for using a Linux based PC will then be minimal and even simpler.


Cross Compiling

Introduction

If you have never cross-compiled anything before, this is the right place for you. If you already have experience and knowledge with cross compilation, this wiki may still be helpful to get you started. If you are looking for information on how to build your own cross-compiler, this place is not for you. (Look HERE instead.)


What this Wiki will cover and not.

- We will use a popular pre-compiled cross-compiler.
- We will work on a Windows (Intel/AMD) based PC.
- All examples herein are based on a UExxES5700 TV set, and should be reproducable on the same.

- We will not cover the compilation of a cross-compiler!
- We will not cover other cross-compilers, operating systems or processor architectures.


A few Questions and Answers:

Q:  What is a cross-compiler?
A:  Basically it is just a compiler built to run on one type of processor (e.g. Intel x86), 
    but which is built and configured for compiling code for another processor (e.g. ARM).

Q:  Should I get a pre-compiled cross-compiler or compile my own?
A:  We hate to waste our time compiling compilers, so always try to find a pre-compiled one!

Q:  Where can I find help to configure my cross-compiler?
A:  Not here. If it's not already in here or in our forums, we don't know.

Q:  Do I need to install the TV specific platform sources? (Such as UExxES6xxx.zip ?)
A: ** It depends on what you need to compile. (See below.)

Q:  Do I have to install the Samsung platform ARM toolchain? (Such as VDLinux-ARMv7-4.4-202-toolchain-v2r2-20110630.tgz ?)
A:  ** Hopefully not, but it depends on what you need to compile. (See below.)

** = unknown and not fact!


Things you need to get started.

WIP! This need checking and adjustment...

1. Install Cygwin for Windows. (Not necessary, but very helpful for the various *nix tools and file utilities.)
2. Install a good text editor (EditPlus, Notepad++ etc.)
3. Download a suitable pre-compiled cross-compiler. 
4. Download your TV kernel sources.
5. Download your TV firmware sources. (?)
6. 



Steps:

(A) Install & Verify the pre-compiled cross-compiler on your PC.
(B) Verify your TV sets processor / architecture. 
(C) Compile "HelloWorld" and run it on TV.
(D) Installing the Samsung Kernel Sources.
(E) Setting up your development environment.
    - Setting up your PATH's + other shell/system variables)
    - Setting up your Makefile
    - other?
(F) Compiling a Kernel Module
(G) Compiling the Kernel

Extras:

(D) Installing the Samsung cross-compiler.

(E) Installing your TV Kernel sources.   
(F) Installing your TV firmware sources.

(G) Compiling the Kernel
(H) Compiling a Kernel module


...

(A) Installing the Cross-Compiler

Go to the Mentor Graphics website, and download the "Sourcery CodeBench Lite Edition" from HERE.
(You'll need to supply an email to get a download link.)
There you will get a few different choices based on the platform. You will have choices such as:

arm-2012.09-63-arm-none-eabi.exe
arm-2012.09-63-arm-none-eabi-i686-mingw32.tar.bz2
arm-2012.09-64-arm-none-linux-gnueabi.exe
arm-2012.09-64-arm-none-linux-gnueabi-i686-mingw32.tar.bz2
arm-2012.09-64-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz2

If you use a x86 Windows based system, choose: "arm-2012.09-64-arm-none-linux-gnueabi.exe".
Run it, and when asked, change the installation directory to something simple like: C:\zarm\csbench
The rest of the installation procedure is self explanatory.

After installation, verify that the cross-compiler PATH variable is properly set and working:

$ arm-none-linux-gnueabi-gcc --version

arm-none-linux-gnueabi-gcc.exe (Sourcery CodeBench Lite 2012.09-64) 4.7.2
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. 

Since this is a Windows installer, Cygwin may or may not catch the updated system PATH variable. Open a new Cygwin shell and check:

$ echo $PATH

If it's not working, you'll have to add the following line in your ~/.bash_profile file.
(On some systems this file is called ".profile".)

PATH=${PATH}:/cygdrive/path/to/csbench/bin;

This should do it.

*******************************************************************************
If you want to have access to the cross-compiler's man pages you'll have to 
add the following line to /etc/man.conf:

 MANPATH_MAP /path/to/csbench/bin /path/to/csbench/share/doc/arm-arm-none-linux-gnueabi/man

and/or possibly this line to your ~/.bashrc

 MANPATH=${MANPATH}:/path/to/csbench/share/doc/arm-arm-none-linux-gnueabi/man;

*******************************************************************************


If you have more questions, check out the Sourcery ARM FAQ.

(B) Determine the TV processor & architecture

In order to properly compile and run anything, you need to know what processor you're programming for. Here is how to find that information.

1. Root your TV and login to a shell. (Instructions HERE.)

2. Verify your TV's processor type and architecture by executing the following at the shell prompt.


shell> cat /proc/cpuinfo

Processor       : ARMv7 Processor rev 0 (v7l)
processor       : 0
BogoMIPS        : 1794.04
processor       : 1
BogoMIPS        : 1794.04
Features        : swp half thumb fastmult vfp edsp neon vfpv3
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x3
CPU part        : 0xc09
CPU revision    : 0 
Hardware        : amber3
Revision        : 0000
Serial          : 0000000000000000


Now, this is not obvious in any way. Problem is that an "ARM" processor is really only a license to manufacture a processor according to ARM Holding's specifications. Therefore any hardware manufacturer can have a processor with an ARM "core". In addition, and to make it even more confusing, an ARM "core" belong to an ARM "architecture" which belong to an ARM "family", whose numbers are little related...

The only way to get some insight is looking at the WikiPedia entries:

1. "List of ARM microprocessor cores"
2. "ARM architecture"

and compare what you find there with the interpretation of the ARM "Features" above, where:


-----------------------------------------------------------------------------
Feature         Description
-----------------------------------------------------------------------------
swp             SWP (SWaP) instruction, which is used to implement a binary semaphore (mutex)
half            Half-precision (16-bit) floating point, "__fp16" data type in gcc
thumb           Thumb instructions
fastmult        Fast multiplication
vfp             Vector Floating Point instruction extension
edsp            Enhanced DSP instructions
neon            NEON SIMD instructions
vfpv3           Vector Floating Point instruction extension Version 3
-----------------------------------------------------------------------------


So from the above we think we have a dual core processor of the ARMv7-R or ARMv7-A architecture. (Of either the Coretx-R or A families, respectively.) But the only NEON enabled processors with both DSP and VFPV3, are the Cortex-A8 and "Cortex-A9 MPCore". But this is not good enough...for a perfectionist. So we search in the ARM on-line documentation for "Main ID Register":


Main ID Register bit functions:

-----------------------------------------------------------------------------
Bits    Field                   Value   Function
-----------------------------------------------------------------------------
[31:24] Implementor             0x41    implementor: ARM
[23:20] Variant                 0x3     variant number or major revision of the processor
[19:16] Architecture            0x7     architecture is given in the feature registers
[15:4]  Primary part number     0xC09   part number: Cortex-A9
[3:0]   Revision                0x0     revision number or minor revision of the processor
-----------------------------------------------------------------------------


Here is a brief list of ARM primary part numbers.


ARM core       CPU part
------------------------
ARM920         0x920
ARM926         0x926
ARM946         0x946
ARM966         0x966

ARM1136        0xb36
ARM1156        0xb56
ARM1176        0xb76
ARM11 MPCore   0xb02

Cortex A5      0xc05
Cortex A8      0xc08
Cortex A9      0xc09
Cortex A15     0xc0f
Cortex R4      0xc14 
Cortex R5      0xc15
------------------------

We finally conclude that our TV processor contains a dual core Cortex-A9 MPCore from the ARMv7-A architecture.

Done!


(C) Compiling "HelloWorld"

We would like to compile our "Hello World" program for our TV.
So create a file like this:

hellow.c

#include <stdio.h>
int main(void) {
       printf("Hello world\n");
       return (0);
}

We now, need to determine what compiler flags to use with our cross-compiler. There are several dozens of compiler options for the CodeSourcery compiler, but we are only interested in the following.


Here are the most important CodeSorcery ARM Cross Compiler options:

-march=                 Specify the name of the target architecture
-mcpu=                  Specify the name of the target CPU
-mfpu=                  Specify the name of the target FPU hardware/format

-marm                   Generate code in 32 bit ARM state.
-mthumb                 Generate code for Thumb state

-mlittle-endian         Assume target CPU is configured as little endian
-mthumb-interwork       Support calls between Thumb and ARM instruction

-mglibc                 Use GNU C library
-muclibc                Use uClibc C library

-static                 Compile and include all static libraries


Here are the choices for the above options:

Known ARM ABIs (for use with the -mabi= option):
  aapcs aapcs-linux apcs-gnu atpcs iwmmxt

Known ARM architectures (for use with the -march= option):
  armv2 armv2a armv3 armv3m armv4 armv4t armv5 armv5e armv5t armv5te armv6
  armv6-m armv6j armv6k armv6s-m armv6t2 armv6z armv6zk armv7 armv7-a armv7-m
  armv7-r armv7e-m ep9312 iwmmxt iwmmxt2 native

Known ARM CPUs (for use with the -mcpu= and -mtune= options):
  arm1020e arm1020t arm1022e arm1026ej-s arm10e arm10tdmi arm1136j-s
  arm1136jf-s arm1156t2-s arm1156t2f-s arm1176jz-s arm1176jzf-s arm2 arm250
  arm3 arm6 arm60 arm600 arm610 arm620 arm7 arm70 arm700 arm700i arm710
  arm7100 arm710c arm710t arm720 arm720t arm740t arm7500 arm7500fe arm7d
  arm7di arm7dm arm7dmi arm7m arm7tdmi arm7tdmi-s arm8 arm810 arm9 arm920
  arm920t arm922t arm926ej-s arm940t arm946e-s arm966e-s arm968e-s arm9e
  arm9tdmi cortex-a15 cortex-a5 cortex-a7 cortex-a8 cortex-a9 cortex-m0
  cortex-m1 cortex-m3 cortex-m4 cortex-r4 cortex-r4f cortex-r5 ep9312 fa526
  fa606te fa626 fa626te fa726te fmp626 generic-armv7-a iwmmxt iwmmxt2 mpcore
  mpcorenovfp native strongarm strongarm110 strongarm1100 strongarm1110 xscale

Known ARM FPUs (for use with the -mfpu= option):
  fpa fpe2 fpe3 fpv4-sp-d16 maverick neon neon-fp16 neon-vfpv4 vfp vfp3 vfpv3
  vfpv3-d16 vfpv3-d16-fp16 vfpv3-fp16 vfpv3xd vfpv3xd-fp16 vfpv4 vfpv4-d16


ARMed with our previous knowledge from part (B) we can try the following:

$ arm-none-linux-gnueabi-gcc.exe -march=armv7-a -mcpu=cortex-a9 -marm -mlittle-endian -mglibc -static hellow.c -o hellows
$ arm-none-linux-gnueabi-gcc.exe -march=armv7-a -mcpu=cortex-a9 -marm -mlittle-endian -mglibc hellow.c -o hellowd

Great! It seem to work. But as you can see, a statically compiled binary is about 100x bigger than a dynamically compiled one of size ~10K. But sometimes we need a static binary as it can help overcome crippled, buggy or platform specific system libraries.

But let's check if we got what we expected:

$ file hellows
hellows: ELF 32-bit LSB executable, ARM, version 1 (SYSV), statically linked, for GNU/Linux 2.6.16, 
not stripped

$ file hellowd
hellowd: ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked (uses shared libs), 
for GNU/Linux 2.6.16, not stripped

$ arm-none-linux-gnueabi-objdump.exe -x hellows |less
...


Looks good, let's try to run it. ( /dtv/usb/sda1 )


ftp <tv_ip> 
ftp> put hellows /tmp/bin/hellows
ftp> put hellowd /tmp/bin/hellowd
ftp> quit

nc <tv_ip> 23 
shell> chmod 777 /tmp/bin/hellow*
shell> hellows
Hello world
shell> hellowd
Hello world

Excellent! We are now ready to try a more advanced example where we will make use of some platform specific system libraries to make a kernel module.


(D) Installing the Samsung Kernel Sources

Download the sources relevant to your TV set from the Samsung Open Source repository.
In our case (UE40ES5700) it would be UExxES6xxx.zip. For other ES models, it would be UNxxES8xxx.zip.


But these two are exactly the same, except the following:

1. E8 and E6 have slightly different VDLinux kernels
2. There is an additional OR1200.ZIP for the E8

The only files (~150) which are different are listed HERE.


After download, do test the downloaded zip archive with:

 $ unzip -t UExxES6xxx.zip

If you are curious to see detailed info of what's inside the ZIP archive before unzipping, do this:

$ unzip -Z UExxES6xxx.zip
...

Extract the ZIP file to a directory where you would like to keep your sources.

$ cd /zarm/src/
$ unzip UExxES6xxx.zip -d UExxES6xxx


This will unzip the following:

alsa-lib-1.0.23.tgz                            Advanced Linux Sound Architecture (audio and MIDI)
ATK.tgz                                        Accessibility Toolkit (screen User Interface)
binutils-2010q1.tgz                            A collection of binary tools (ld, as, nm, objdump etc.)
BROADCOM-bthid.tgz                             Broadcom Bluetooth HID drivers (keyboards, mice, game controllers)
BROADCOM-btusb.tgz                             Broadcom Bluetooth USB drivers (keyboards, mice, game controllers)
busybox-1.18.1.tgz                             Busybox combines many common UNIX utilities into a single executable
Cairo.tgz                                      A 2D graphics library (X Window, quartz, win32, PDF, PS, SVG file output)
FFMPEG.tar.gz                                  A cross-platform solution to record, convert and stream audio and video
glibc-2.11-2010q1.tgz                          The GNU C Library
Glibmm.tgz                                     A C++ interface for the popular cross-platform library Glib
gnutls-2.6.4.tar.gz                            GNU Transport Layer Security Library (SSL, TLS and DTLS protocols)
iptables-1.4.10.tgz                            Linux kernel firewall
libgcrypt-1.4.5.tar.gz                         A general purpose cryptographic library 
libgpg-error-1.7.tar.gz                        A library that defines common error values for all GnuPG components
LIBGPHOTO2.tar                                 The core library designed to allow access to digital camera by external programs
LibMMS_0.6.2.tgz                               A library for parsing mms:// and mmsh:// type network streams
libsoup.20120109.tgz                           An HTTP client/server library for GNOME
libtasn1-2.5.tar.gz                            The ASN.1 library used by GnuTLS
LIBUSB.tar                                     A C library that gives applications easy access to USB devices
Pango.tgz                                      A library for layout and rendering of multi-language text
RALINK_RTNET5572STA_V_2_5_0_1.tgz              Ralink RTnet RT5572 (Wifi USB dongle drivers)
RALINK_RTUTIL5572STA_V_2_5_0_1.tgz             Ralink RTnet RT5572 (Wifi USB dongle utilities)
readme.zip                                     HOW_TO_BUILD_X9X10.txt
SDL.tar.gz                                     Simple DirectMedia Layer (a multimedia library written in C)
uvc.tar.gz                                     USB Video Class (streaming webcams, digital cameras etc)
v4l2.tar.gz                                    Video4Linux-2 (a video capture API for Linux)
VDLinux_2.6.35.11.tgz                          Kernel sources (VDLinux, Tuxera NTFS, RFS, LinuStoreIII, FSR)
webkit-gtk.20120109.tgz                        WebKit is an open source web browser engine (Safari, Chrome)
WIRELESSTOOLS_29.tgz                           Wireless Tools (iwconfig, iwlist, etc)
xfsprogs-3.1.5.tgz                             A set of command-line tools to manage XFS filesystems


Which of these do you really need? Now that is the million dollar question. The simple and stupid answer is, that it depends on what you want to do.


a) If you need to compile some simple C-code using standard clibs and linux 
   system calls, you probably don't need any. (E.g. helloworld.c)

b) If you need to compile your own Kernel module (hellok.ko) you probably need:

   VDLinux_2.6.35.11.tgz

c) If you want to compile your own Kernel image (uImage) you probably only need:

   VDLinux_2.6.35.11.tgz

d) If you want to compile your own library object (somelib.so) you probably only need:

   VDLinux_2.6.35.11.tgz
   ++++

e) If you need to compile your own device driver you probably need:

   <the device driver sources>
   VDLinux_2.6.35.11.tgz
 ? glibc-2.11-2010q1.tgz
 ? Glibmm.tgz

f) If you want to compile your own TV DSP (exeDSP, micom etc) you're screwed 
   (by Samsung) since there are no publicly available sources for that.


The better answer is, that it depends on how your kernel image has been setup and how you intend setup your compilation environment. We really don't want to have to setup and compile all sources from scratch, just to make a simple kernel module. The way to do that is by telling your cross-compiler where to find the kernel header files that it has to use the same configuration as that used to compile your kernel.

So for further details, we look at HOW_TO_BUILD_X9X10.txt inside the readme.zip. This file explains, in a very screwy way, how to build each of the items included on the UExxES6xxx.zip file. In fact you should not rely blindly on this info.

For example:

...
[ Building linux kernel ]

* Source code name : VDLinux_2.6.35.11.tgz 
* Unpack the kernel tarball and cd into it.
* Run "cd VDLinux_2.6.35.11/linux-2.6.35.11/".
* Run "cp -ar arch/arm/configs/X10P_defconfig_release .config".
* Run "make oldconfig".
* Run "make uImage".
 
[ Building busybox ]

* Source code name : busybox-1.18.1.tgz
* Unpack the busybox tarball and cd into it.
* Run "make clean". 
* Run "cp -ar configs/busybox_config .config".
* Run "make ARCH=arm CROSS_COMPILE=arm-v7a8v2r2-linux-gnueabi- oldconfig".
* Run "make ARCH=arm CROSS_COMPILE=arm-v7a8v2r2-linux-gnueabi- CONFIG_PREFIX=../temp_rootfs install".
* Run "cd ../temp_rootfs/bin".
...


Clearly, for general purpose use we need at least the VDLinux sources installed.
So we extract this in the same directory with:

$ (tar -zxvf VDLinux_2.6.35.11.tgz >vdlinux_tar.log) 2>&1


Here we have redirected the output to a log file for reference, while any errors will be shown on screen. This will create VDLinux_2.6.35.11 with the following sub-directories:

linux-2.6.35.11
TUXERA_NTFS
RFS_3.0.0_b043-LinuStoreIII_1.2.0_b039-FSR_1.2.1p1_b139_RTM

The last two directories are soft linked into the ./tntfs and ./rfs sub-directories of ./linux-2.6.35.11. However, these links are not well re-made (especially after decompression). So if you think you'll need them, it is probably better to just copy them into there...

< more details needed >


WIP!!


placeholder



(E) Building the Kernel using Cygwin

If you're not using Cygwin, you can skip directly to: Cross-compiling_(ES_series)#Compiling_the_Kernel




!! STOP !! First read these REQUIREMENTS!!





Before proceeding, read the following very carefully! I mean it!

We strongly discourage the use of Mentor Graphics (Aka. CodeSourcery)
CodeBench Lite ARM cross-compilers, for anything other than compiling
simple freestanding programs. If you intend to write a simple kernel module or
kernel device driver, or any other less trivial development, you "must" use a
native linux environment. Although Cygwin provides for an almost-linux
environment, the quirks introduced by mixing the various Windows-Cygwin-Linux
tools, creates a huge headache, that most people should avoid.


Below we will show you that you can indeed compile a Linux Kernel using Cygwin
and some of the tools mentioned above. But the road to get there is a horrible
waste of time and energy, trying to patch and resolve problems that should not
be there in the first palce. Do not bother to ask in the forums for support
using Cygwin or CodeBench + Windows combination. You have been warned!

Instead follow the instructions: Cross-compiling VM-style (ES)

REQUIREMENTS!!

If you intend to use Cygwin with the Sourcery CodeBench (a pre-compiled windows executable) ARM cross-compiler, do not believe for one second, that you will be successful compiling anything other than HelloWorld.c for your Linux based ARM (e.g. Android phone, SmartTV etc.) device, unless you follow these requirements.

First let me tell you what the main problems are with using windows.

Standard Windows (XP, Vista, Windows-7):

 - Uses ACL to set file permissions and ownership (Not Linux compatible.)
 - Has it's own way of creating symbolic links (Not Linux compatible.)
 - Uses a non case-sensitive default for its fixed NTFS drives. 
   (Often and silently break archives originally compressed under Linux.)
 - Uses the Win32 (non POSIX) standard for file paths (Not Linux compatible.)
 - Uses the 2 characters Carriage-Return and New-Line ("\r" & "\n") for
   End-of-Line (EOL) representation, contrary to POSIX, which uses only NL.

If you're going to compile a Linux Kernel on Cygwin, you have to first resolve
the above Windows problems. Second, you'll need to be aware of other Cygwin quirks,
that could affect your success. Here is how.

 (a)    "Disable" the native Windows use of Access Control Lists (ACL).
	This is probably not completely correct/possible, but by making yourself 
	a permanent Administrator, it will have the same effect. If you have ever 
	owned a Vista machine, you have probably already done this, as it created 
	huge problems in the past. Google it! Apparently Cygwin has a bug that 
	doesn't take into account default ACL settings and "umask" gives the wrong
	permissions. So you have to change permissions and ownerships manually
	with chmod/chown. There is one other possibility (not verified). You can 
	try to mount your Cygwin partitions in /etc/fstab with the "noacl" flag: 
	
	none /cygdrive cygdrive binary,noacl,posix=1,user 0 0
	
	But see below first. If you do insist on messing with Windows ACL's check 
	out commands like: "getfacl, setfacl, cacls" and to recursively find and 
	set all Users files ACLs to "rwx", use: 
	"find /cygdrive/c/Users -exec setfacl -r -m default:other:rwx {} \;"
	Or read all about it here: http://tiny.cc/pr9yqw & http://tiny.cc/rwczqw


 (b)    Enable NTFS case-sensitivity by editing the Windows registry. 
	We are constantly warned by MS for unexpected results of doing this, and 
	that it could have grave consequences for your system. That is complete 
	BS, as the only unexpected consequence is that you start wondering why you 
	didn't re-enable this idiotic Windows "feature" ages ago. I certainly 
	questioned what the fcuk was going on every time I extracted some Linux 
	tarball, and kept getting dozens of files having the "same" name and given 
	the "choice" of renaming them! (WTF! They already had different names, as
	Ab.txt =/= aB.txt !) If you choose not to "rename" them, only the last one 
	extracted will survive! You will never know! Do this, in your Cygwin Bash 
	shell: 
	
	$ regedit.exe &
	--> navigate to: [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\kernel]
	--> add the following item:  obcaseinsensitive REG_DWORD 0
	
	Now reboot and and let your PC "settle", by running some of your 
	favorite apps. No problems? Good, reboot again and you're done.
	
	NOTE: Cygwin online documentation claim two things that are not true:
	
	   (1) that you can temporarily remount a disk as "posix=1" to enable
	   case sensitivity. This doesn't work, and even if it did, you
	   would immediately loose this, after closing last Cygwin shell.
	   It only works if NTFS case-sensitivity is already enabled, but 
	   have been disabled when default mounted in /etc/fstab. 
	   (2) "mount" only shows disk as "posix=0" (case insensitive), 
	   and never as "posix=1" (case sensitive), as claimed by online 
	   documentation.
	
	So until you have implemented a case sensitive system, avoid developing
	Linux systems using tarballs extracted under windows.
	
 (c)    Windows have it's own screwed up way of doing symbolic links, that are 
	not compatible with linux symbolic links. A Win32 symlink is just a file
	with a special tag followed by the file inode address AFAIK. But Cygwin 
	can't handle this very well, so that if you symlink a regular file, your 
	Cygwin application will fail, reading the content, instead of following 
	the link. But if you symlink a directory, it works. This is probably 
	a Cygwin bug. But it doesn't matter, because if you use original symlinks
	(from a Linux tarball, for example) you will get fcuked by the Win32 API 
	anyway, regarding both file permissions and ownerships. 
		
	Instead, just find and delete all symlinks, and try to recreate them 
	within Cygwin. But to be sure, just copy the files/directories instead.
	(This is specially important if you try to compile linux kernels with 
	thousands of files, some which may be symlinked.)
	
	To find all symlinks use:  "find ./ -type l"
	
	So NEVER extract any source code tarballs using Windows utilities such 
	as Winzip, 7zip etc, as they will fail to re-create proper hard links.
	

 (d)    One major obstacle, when trying to use a (native windows) pre-compiled 
	cross-compiler with Cygwin, is that they use Win32 file paths. These are
	not normally compatible with Linux based tools like binutils and make, 
	which require POSIX paths. Cygwin has it's own internal mechanism for 
	dealing with Win32 paths, through /bin/cygpath.exe. This program can 
	convert back and forth. But the problem occurs when POSIX programs like 
	"make" is using windows cross-compilers (e.g. Sourcery CodeBench Lite) 
	which return results with Win32 paths. According to their documentation
	this can be handled by defining the shell variable "CYGPATH=cygpath".
	However, this is not working as described and expected, as easily 
	demonstrated with:
	
	  $ arm-none-linux-gnueabi-gcc.exe --print-sysroot
	  d:\zarm\csbench\bin\../arm-none-linux-gnueabi/libc
	
	So you might think that adding the "--sysroot" flag might help. Well it 
	does somewhat: but not completely:
	
	  $ arm-none-linux-gnueabi-gcc.exe --sysroot=/cygdrive/d/zarm/ -print-sysroot
	  /cygdrive/d/zarm/
	
	becuase if you do the same with "-print-file-name" which is used in the 
	top Makefile, you still get a Win32 path!
		
	  $ arm-none-linux-gnueabi-gcc.exe -print-file-name=include
	  d:/zarm/csbench/bin/../lib/gcc/arm-none-linux-gnueabi/4.7.2/include
		
	But apparently this is not all. It seem that CodeBench is also "leaking" 
	out other internal variables that are poisoning the environment with Win32
	pathnames that have been compiled in. For example, if you do the following: 
		
		arm-none-linux-gnueabi-gcc.exe -v --help >/dev/null
	
	You'll get things like: 
	
		COLLECT_GCC=D:\zarm\csbench\bin\arm-none-linux-gnueabi-gcc.exe
		COLLECT_LTO_WRAPPER=d:/zarm/csbench/bin/../libexec/gcc/arm-none-linux-gnueabi/4.7.2/lto-wrapper.exe
		COLLECT_GCC_OPTIONS
		COMPILER_PATH
		LIBRARY_PATH
	
	and whole bunch more. But these are not a problem...until they start popping 
	up in your assembly code. The most interesting of these are the "-iprefix" 
	and "-isysroot" which seem to propagate into assembler without anyone ever 
	entering these. For example, in the file ./kernel/bounds.s we find:
	
		@ -iprefix d:\zarm\csbench\bin\../lib/gcc/arm-none-linux-gnueabi/4.7.2/
		@ -isysroot /cygdrive/d/zarm/csbench/arm-none-linux-gnueabi/libc
		...		
		@ -isystem /cygdrive/d/zarm/csbench/lib/gcc/arm-none-linux-gnueabi/4.7.2/include
		@ -include include/generated/autoconf.h -MMD kernel/.bounds.s.d
	
	The only soultion is to manually specify new paths for both -sysroot and -iprefix.
	If you use both this option and the -isysroot option, then the -sysroot option 
	will apply to libraries, but the -isysroot option will apply to header files. 
	We do this in the Makefile.


 (e)    Then we have the issue with differing End-of-Line (EOL) character(s) on POSIX 
	based systems like Cygwin (and Linux) and that used by windows. Where Win32 uses 
	the 2 character combination of a Carriage-Return and a New-Line ("\r" & "\n") 
	for EOL representation, POSIX ony use NL ("\n"). This has serious consequences
	for the in-between compilation, where GCC is automatically creating link scripts
	to be used later in the linking process. Thus the linking scripts need to be 
	converted. This is done by the "unix2dos" and "dos2unix" utilities. But we 
	have to patch the Makefiles to use these. 

	
 (f)	The final Cygwin quirk relates to the automatic recognition of .exe files.
	As you know, Cygwin treats .exe files on equal footing as non-.exe files,
	almost! Basically if you have a script called "myprog" and an executable
	called "myprog.exe", then "myprog.exe" takes precedence when typing "myprog".

	http://cygwin.com/cygwin-ug-net/cygwin-ug-net.html
	http://seit.unsw.adfa.edu.au/staff/sites/hrp/webDesignHelp/cygwin-ug-net-nochunks.html#USING-CYGWINENV

	 "Executable program filenames end with .exe but the .exe need not be included in 
	 the command, so that traditional UNIX names can be used. However, for programs 
	 that end in ".bat" and ".com", you cannot omit the extension."

	As a side effect, the ls filename gives information about filename.exe if 
	filename.exe exists and filename does not. In the same situation the function 
	call stat("filename",..) gives information about filename.exe. The two files can 
	be distinguished by examining their inodes, as demonstrated below. 

	If a shell script myprog and a program myprog.exe coexist in a directory, the 
	program has precedence and is selected for execution of myprog.

	The gcc compiler produces an executable named filename.exe when asked to produce 
	filename. This allows many makefiles written for UNIX systems to work well under 
	Cygwin.

	Unfortunately, the install and strip commands do distinguish between filename 
	and filename.exe. They fail when working on a non-existing filename even if 
	filename.exe exists, thus breaking some makefiles. This problem can be solved by 
	writing install and strip shell scripts to provide the extension ".exe" when 
	needed.

Did you read and understand all that? Good, then you are ready to patch the Supplied Samsung kernel.

I have actually patched it for you, but you should read through to see what was done. (Download link at the end.)

Setting up your Cygwin environment

The BASH Shell environment

We need to make sure our Cygwin Bash shell environment is as close as possible to what you'd find on a typical Linux distribution. The way to find out what those settings are is to issue the following command, which will display the enabled and disabled Bash shell environment settings. contained in the $BASHOPTS and $SHELLOPTS environment variables.

set -o && echo -e "\n" && shopt -p 

The $BASHOPTS and $SHELLOPTS environment variables only show the enabled ones.

$ echo "SHELLOPTS=$SHELLOPTS"
SHELLOPTS=braceexpand:emacs:hashall:histexpand:history:interactive-comments:monitor

$ echo "BASHOPTS=$BASHOPTS"
BASHOPTS=checkwinsize:cmdhist:expand_aliases:extglob:extquote:force_fignore:histappend:
         interactive_comments:login_shell:progcomp:promptvars:sourcepath

Make them match these, and you should be ok.


The Cygwin Specific variables

Next you should set these two variables.

export CYGWIN=nodosfilewarning
export CYGPATH=cygpath

The first one disables the Warning message that appears when Cygwin detects files using DOS EOL. While the second sets the path used for CodeBench to find the cygpath.exe path conversion program.

The Compiler Specific variables

Then to avoid having to type very long command lines for just doing a "make", we define the following:

export ARCH=arm 
export CROSS_COMPILE=arm-none-linux-gnueabi-
export CONFIG_CROSS_COMPILE=${CROSS_COMPILE}
export KCFLAGS="-imultilib ./ -mno-unaligned-access -mlittle-endian \
                -marm -mapcs -mabi=aapcs-linux -mno-thumb-interwork \
                -march=armv7-a -mfloat-abi=softfp -mtune=cortex-a8 -mfpu=vfp3"

The setup script

All of the above (and more) is put in our setup script.

sexport.sh:

#! /bin/sh
# sexport.sh - Exports variables used by the CodeSorcery ARM cross-compiler 
#==============================================================================
# NOTE: Export doesn't work when used in shell script. It can only be "sourced"
#       So you need to run this with:  ". ./sexport.sh"
#------------------------------------------------------------------------------
checkwindrives () {
 MYOS=`uname -o`
 if [[ $MYOS == "Cygwin" ]]; then 
	 TEST=`mount |grep -i "posix=0"`
	 if [ $? != 1 ]; then
	    echo -e "One or more of your harddrives are mounted as NOT case-sensitive!\n";
	    echo -e "$TEST\n";
	    echo "A case sensitive NTFS drive may be necessary for compiling Linux Kernels on Cygwin."; 
	    echo "You can fix this by enabling case sensitivity in the windows registry and rebooting:";
	    echo "  HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\kernel\ DWORD:obcaseinsensitive 0";
	    echo "You can also try to remounting cygdrives in 'posix=1' mode by editing /etc/fstab to:";
	    echo "  'none /cygdrive cygdrive binary,posix=1,user 0 0'"
	    echo "or try directly on command line (after reboot) with:";
	    echo "  'umount /cygdrive/d'";
	    echo "  'mount -f -o binary,posix=1 d: /cygdrive/d'";
	    echo "WARNING: Doing this may have unknown consequences for other native windows programs.";
	    echo "    See: http://tiny.cc/4s0uqw and http://tiny.cc/ik1uqw and http://tiny.cc/4oavqw ";
	 fi
 fi
}
if [ $0 != "bash" ]; then
	echo "This export script need to be sourced!";
	echo "That is, you have to run it with: . ./sexport.sh"
	exit 1;
else
	export ARCH=arm 
	export CROSS_COMPILE=arm-none-linux-gnueabi-
	export CONFIG_CROSS_COMPILE=${CROSS_COMPILE}
	export PATH=${PATH}:/cygdrive/d/zarm/devbin/bin:.
	export SAMYGO=arm-samygo-linux-gnueabi-
	export SYSROOT=/cygdrive/d/zarm/csbench/arm-none-linux-gnueabi/libc

	export KCFLAGS="-imultilib ./ -mno-unaligned-access -mlittle-endian \ 
			-marm -mapcs -mabi=aapcs-linux -mno-thumb-interwork \ 
			-march=armv7-a -mfloat-abi=softfp -mtune=cortex-a8 -mfpu=vfp3"

	MODHOME=/cygdrive/d/zarm/myarm/kernel_modules
	export INSTALL_PATH=${MODHOME}/kimagemap
	export MODLIB=${MODHOME}/kmods
	export INSTALL_MOD_PATH=/

	shopt -s checkwinsize
	shopt -s extglob
	shopt -u hostcomplete

	echo -e "\nExported:\n "
	echo " ARCH=$ARCH"
	echo " CROSS_COMPILE=$CROSS_COMPILE"
	echo " CONFIG_CROSS_COMPILE=$CONFIG_CROSS_COMPILE"
	echo " SAMYGO=$SAMYGO"
	echo;
	echo " GCC_EXEC_PREFIX=${GCC_EXEC_PREFIX}";
	echo " SYSROOT=${SYSROOT}";
	echo " LD_RUN_PATH=${LD_RUN_PATH}";
	echo;
#	echo " KCPPFLAGS=$KCPPFLAGS"
	echo " KAFLAGS=$KAFLAGS"
	echo " KCFLAGS=$KCFLAGS"
	echo;
	echo " INSTALL_PATH=${INSTALL_PATH}"
	echo " INSTALL_MOD_PATH=${INSTALL_MOD_PATH}"
	echo " MODLIB=${MODLIB}"
	echo;
	echo "You can use: 'export -n <VAR>'  OR:  'unset <VAR>'  to remove property.";
	echo "Also confirm that 'alias'es are not masking any important commands. Use 'unalias' to fix.";
	echo;
#	echo " SHELL=${SHELL}";
	echo " CYGPATH=${CYGPATH}";
	echo " CYGWIN=${CYGWIN}";
	echo;
	echo "Enabled BASH(OPTS) environment variables:";
	echo -e "\n ${BASHOPTS}\n" |sed -r 's/\:/\n /g'
	echo "Enabled SHELL(OPTS) environment variables:";
	echo -e "\n ${SHELLOPTS}\n" |sed -r 's/\:/\n /g'
	echo "Use 'set +o <shell_opt>' and 'shopt -u <bash_opt>' to disable.";
	echo "Use 'set -o <shell_opt>' and 'shopt -s <bash_opt>' to enable.";
	echo;
	
	# Check locally mounted drives for case sensitivity (Cygwin only):
	checkwindrives
fi

Obviously you have to edit the paths to what you have...

Patching the Kernel for Cygwin use

We will not show the boring details for this patching here.
Please have look in our forum HERE for all the gory details.

Since there is no available info how to patch VDLinux (based on 2.6.35.13) for Cygwin compilation, the patches used to get this working was handpicked and applied by hand from the sources below. It is not clear as to what patches are actually needed, and others may overlap in their functionality.

The patches are based on the following:

[1] https://sourcery.mentor.com/GNUToolchain/kbentry21
[2] http://communities.mentor.com/community/cs/archives/arm-gnu/msg01267.html
[3] http://communities.mentor.com/community/cs/archives/arm-gnu/msg01799.html

[4] HOWTO: Linux kernel compilation for ARM using cygwin (01 Aug 2007)
[5] HOWTO: cross-compile the Linux kernel on Cygwin (11 Jun 2012)
[6] Cygwin User guide

[7] Google: "cygwin kernel compile error site:communities.mentor.com/community/cs/archives/arm-gnu/"
[8] Patch for Toolchain 4.6.3-cygwin (perhaps not needed)

Download and install the patched VDLinux

NOTE: This is not the same as the "Patched Firmware" in the top E-Series Wiki page.

This patch is only for use with Cygwin + CodeBench Lite (windows) cross compiler!!


blah blah

(F) Compiling the Kernel

Before you build your own kernel module, it will help to first make sure you can build a kernel image. Although this is not necessary, it will help with setting up some required automatically generated scripts. These scripts generate some special header files that contain the Linux kernel versions and "vermagic" that might be needed for your kernel module to work.


Checking your Kernel version

The special files with version information are:

./include/linux/version.h:
#define LINUX_VERSION_CODE 132643
#define KERNEL_VERSION(a,b,c) (((a) << 16) + ((b) << 8) + (c))

./include/generated/utsrelease.h:
#define UTS_RELEASE "2.6.35.13"

./include/linux/vdlp_version.h:		(is symlinked to: vdlp_version_X10P.h )
#define DTV_KERNEL_VERSION "0062, release"
#define DTV_LAST_PATCH "0713, DTV, X10P, release, AUS_BRANCH"

Find a kernel module from either the TV itself or from downloading and extracting the firmware. Then find your current Kernel version by:

strings <some-module>.ko |grep "vermagic" 

For example:

$ strings drivers/hid/hid.ko |grep "vermagic"
vermagic=0062, release SMP preempt mod_unload ARMv7

Then edit the file above by changing the DTV_KERNEL_VERSION value to what you found.





(G) Compiling a Kernel module

Introduction

Linux is a monolithic kernel, where all of the code and data that makes up the
image are linked into one binary and loaded into memory.

"A very useful part of the Linux kernel architecture is the support for
loadable kernel modules. These modules allow the otherwise monolithic kernel
to be split up into smaller components that can later be loaded as required,
allowing the kernel to ship with support for a wide range features but only
load those that are needed.

Kernel modules also ease the development of new features such as file systems
or device drivers, as a new experimental modules can be quickly built, loaded
into a basic kernel, exercised, and then unloaded. This is much faster then
the build, flash, restart process that would otherwise be required."

The main reason for this, is that you don't have to re-compile the entire
kernel from the full kernel source tree. It should be enough to just have
the kernel header files, in order to build a module.

Internally a module is a standard ELF executable file with a .ko extension and
a few special sections such as .modinfo for the module metadata and .init.text
for the module initialization code. (This is normally done by linking
yourprogram.o with vermagic.o, which is transparent to the developer.)
A nice thing about modules being ELF files is that they can be generated
and inspected by standard tools.


In the current 2.6 series, the ARM kernel is laid out as follows:

Start           End             Contents
-----------------------------------------------------------------------------
0xFF000000      0xFFFFFFFF      Vector page, DMA region, and others
VMALLOC_END     0xFEFFFFFF      free
VMALLOC_START   VMALLOC_END     vmalloc() / ioremap() space
PAGE_OFFSET     high_memory     The Linux kernel
TASK_SIZE       PAGE_OFFSET-1   Kernel module space (16 MB)
0x00001000      TASK_SIZE-1     User space (~3 GB)
0x00000000      0x00001000      Vector page / Null pointer trap
-----------------------------------------------------------------------------
PAGE_OFFSET = 0xC0000000
TASK_SIZE   = 0xBF000000

Note that these are virtual addresses, which are different than the physical address space of the board.


The HelloWorld-1 kernel module

We will attempt to build two kernel modules. One very basic in "Hello World" style to see that it compiles and works,
and another only slighlty more complicated to check if kernel debugfs works.

Any module which want to send info to kernel messages ( /proc/kmsg ) have to include the kernel.h header file.
What is actually shown depend on the current kernel debug level as set in /proc/printk...

./include/linux/kernel.h:

...
#define KERN_EMERG      "<0>"   /* system is unusable                   */
#define KERN_ALERT      "<1>"   /* action must be taken immediately     */
#define KERN_CRIT       "<2>"   /* critical conditions                  */
#define KERN_ERR        "<3>"   /* error conditions                     */
#define KERN_WARNING    "<4>"   /* warning conditions                   */
#define KERN_NOTICE     "<5>"   /* normal but significant condition     */
#define KERN_INFO       "<6>"   /* informational                        */
#define KERN_DEBUG      "<7>"   /* debug-level messages                 */
...

You can see your current kernel debug level with either:

shell> sysctl -a
...
kernel.printk=
... 
OR 
shell> cat /proc/loglevel	???		<==  check!!
shell> cat /proc/printk
kernel.printk = 4  4  1  7

You can read more about these here:

modinfo -p ${modulename}
echo -n ${value} > /sys/module/${modulename}/parameters/${parm}


Now, to the actual kernel module code.

hellok1.c:

#include <linux/module.h>  /* Needed by all modules */
#include <linux/kernel.h>  /* Needed for KERN_ALERT */

MODULE_LICENSE("GPL");
MODULE_AUTHOR("E:V:A 2013");
MODULE_DESCRIPTION("Demo kernel module for MST-X10P (ARM Cortex A9)");

int init_module(void) {
   printk(KERN_ALERT "E:V:A is in the Kernel!\n");
   return 0;
}

void cleanup_module(void) {
  printk(KERN_ALERT "Goodbye TV Kernel!\n");
}


Now that was the trivial part!

The Makefile

The most difficult part of compiling a kernel module, is setting up the Makefile that contain all the compilation instructions, locations and parameters needed for your cross-compiler. It is good to be familiar with the "Makefile" language, as it is tab and space sensitive, and have many other pitfalls, that can easily be overseen. The Makefile is also closely connected with how your kernel have been compiled, so if you're missing kernel support for your modules features (e.g. debugfs), you will not get anything... (It should be noted that Makefile may also contain the functionality of Kbuild, which is very similar, but whose structure is even more simple. We will not cover the details of this here.)


Makefile:

ifneq ($(KERNELRELEASE),)
	obj-m += hellok1.o
else
	KERNELDIR := /your/path/to/linux-2.6.35.11/
all:
	$(MAKE) -C $(KERNELDIR) M=$(PWD) modules
clean:
	rm -fr ./.tmp_versions Module.symvers modules.order ./.*.cmd ./*.mod.*
	ls -al
endif

NOTE: The code above contains hidden tabs, so make sure they're still there if you decide to copy & paste the code from above!


Compiling the module

Go to your directory containing your Makefile and module programs.

Minimally containing:

Makefile
hellok1.c
hellok2.c

Then just type "make". (Make sure you have correctly setup your BASH environment variables.)

Inserting the Module

Copy the module to your TV in the same way we did in the beginning.

For our TV set we have in dmesg:

   ...
    SAMSUNG VDLP Kernel
    Version : 0064, release
    Applied Last Patch Number : 0716, DTV, X10P, release, DEU_BRANCH
   ...

So if we try to insert a kernel module with the wrong vermagic, we get an error message in "dmesg":

   shell> /tmp/bin/busybox insmod /dtv/usb/sda1/hellok1.ko
   shell> dmesg
   ...
   hellok1: version magic '0062, release SMP preempt mod_unload ARMv7 ' should be '0064, release SMP preempt mod_unload ARMv7 '
   ...

We edit the version.h file (mentioned above) and recompile our module. This time "dmesg" greets us with:

   ...
   E:V:A is in the Kernel!
   hellok1 mod ld
   ...

You can then see your module in the FS:

   shell> ls -l /sys/module/hellok1
   drwxr-xr-x    2 root     0                0 Jan  1 00:41 holders
   -r--r--r--    1 root     0             4096 Jan  1 00:41 initstate
   -r--r--r--    1 root     0             4096 Jan  1 00:41 refcnt
   -r--r--r--    1 root     0             4096 Jan  1 00:41 srcversion
   shell> cat /sys/module/hellok1/initstate
   live

To remove your module from the kernel use:

   shell> rmmod hellok1

In "dmesg":

   ...
   Goodbye TV Kernel!
   hellok1 mod uld
   ...

If you try to load the hellok2.ko module on a linux kernel configured without DEBUGFS, you'll get this message (dmesg):

   ...
   hellok2: Unknown symbol debugfs_remove_recursive (err 0)
   hellok2: Unknown symbol debugfs_create_file (err 0)
   hellok2: Unknown symbol debugfs_create_dir (err 0)
   ...


The HelloWorld-2 kernel module

Now that we have gained some intellectual kernel meat, we can design a slightly more useful kernel module. One that actually does something.

The following code is an example kernel module that shows the life cycle and provides a function to user space over debugfs. It creates a new file in /sys/kernel/debug/hello/ping that can be opened and read from user space. Reading this file gives a short message that includes the number of times the file has been opened. All original comments have been removed, but can be found in [3].


hellok2.c:

#include <linux/init.h>
#include <linux/module.h>
#include <linux/debugfs.h>
#include <linux/seq_file.h>
MODULE_LICENSE("Dual BSD/GPL");

static struct dentry *root_dir;	
static int calls;
static int hello_print(struct seq_file *s, void *p) {
	seq_printf(s, "Called %d times\n", ++calls);
	return 0;
}

static int hello_open(struct inode *inode, struct file *file) {
	return single_open(file, hello_print, inode->i_private);
}

static const struct file_operations hello_fops = {
	.open = hello_open,
	.write = NULL,
	.read = seq_read,
	.llseek = seq_lseek,
	.owner = THIS_MODULE,
};

static int hello_init(void) {
	printk(KERN_ALERT "Hello world\n");
	root_dir = debugfs_create_dir("hello", NULL);
	debugfs_create_file("ping", 0444, root_dir, NULL, &hello_fops);
	return 0;
}

static void hello_exit(void) {
	debugfs_remove_recursive(root_dir);
	printk(KERN_ALERT "Goodbye Dear Module\n");
}

module_init(hello_init);
module_exit(hello_exit);

The Makefile for this is your homework, but a good hint is that the tricky part this time, is not the Makefile, but enabling DEBUGFS in your kernel, if not already enabled...


Enabling DEBUGFS in Kernel

Enter the kernel configuration menu:

make menuconfig

and enable the following:

     (a) Go to the "Kernel Features" section.
         Enable ARM EABI:

            CONFIG_AEABI=y

     (b) Go to the "Kernel hacking" section.

        1. Enable the Debug Filesystem:

            CONFIG_DEBUG_FS=y

        2. Enable Kernel debugging. The screen will change and a
           number of new, debug-related options will come visible.

            CONFIG_DEBUG_KERNEL=y

        3. Enable Compile the kernel with debug info. This turns on the
           standard -g option to the compiler and linker and expands the
           vmlinux image to include all the symbolic, line-level debugging
           information needed later.

            CONFIG_DEBUG_INFO=y

Note: (2) and (3) are not recommended for a production kernels.

Save and exit. Then recompile your kernel.



WIP!! ....

References

CodeSourcery Application Notes:

[1] "Flying Introduction to Linux Kernel Development" (AN001)
[2] "Using Sourcery CodeBench to Debug the Linux Kernel" (AN002)
[3] "Using Sourcery CodeBench to Develop and Debug a Linux Kernel Module" (AN003)


[4] "GNU make manual"
[5] "Kernel.org Makefile manual"
[6] "seq_file HowTo"

Common Make Variables

Some commonly-used predefined "make" variables:

AR              default 'r'           Archive-maintaining program
AS              default 's'           Program for compiling assembly files
CC              default 'c'           Program for compiling C programs
CXX             default '++'          Program for compiling C++ programs
CPP             default '(CC) -E'     Program for running the C preprocessor
FC              default '77'          Program for compiling Fortran and Ratfor programs
M2C             default '2c'          Program to use to compile Modula-2 source code
PC              default 'c'           Program for compiling Pascal programs
CO              default 'o'           Program for extracting a file from RCS
GET             default 'et'          Program for extracting a file from SCCS
LEX             default 'ex'          Program to use to turn Lex grammars into source code
YACC            default 'acc'         Program to use to turn Yacc grammars into source code
LINT            default 'int'         Program to use to run lint on source code
MAKEINFO        default 'akeinfo'     Program to convert a Texinfo source file to Info file
TEX             default 'ex'          Program to make TeX dvi files from TeX source
TEXI2DVI        default 'exi2dvi'     Program to make TeX dvi files from Texinfo source
WEAVE           default 'eave'        Program to translate Web into TeX
CWEAVE          default 'weave'       Program to translate C Web into TeX
TANGLE          default 'angle'       Program to translate Web into Pascal
CTANGLE         default 'tangle'      Program to translate C Web into C
RM              default 'm -f'        Command to remove a file
# ------------------------------------------------------------------
ARFLAGS         Extra flags to the archive-maintaining program; default 'rv'.
ASFLAGS         Extra flags to the assembler when invoked on a '.s' or '.S' file.
CFLAGS          Extra flags to the C compiler.
CXXFLAGS        Extra flags to the C++ compiler.
COFLAGS         Extra flags to the RCS co program.
CPPFLAGS        Extra flags to the C and Fortran preprocessor
FFLAGS          Extra flags to the Fortran compiler.
GFLAGS          Extra flags to the SCCS get program.
LDFLAGS         Extra flags to compilers when they invoke the linker, 'ld'.
LFLAGS          Extra flags to Lex.
YFLAGS          Extra flags to Yacc.
PFLAGS          Extra flags to the Pascal compiler.
RFLAGS          Extra flags to the Fortran compiler for Ratfor programs.
LINTFLAGS       Extra flags to lint.

(Z) Place Holder Template

Bah!

Testing:

 
Some ''whacky'' code.
Next line.

Testing more:

Some whacky code.
	LINK
last line

ok