All the links below are free high level languages and Mathematics Packages for Scientific Computation on Linux. These packages are usually like a Mathematical Laboratory in which numerical computations can be done and usually have their own interpreted language. They either link to a popular (free) plotting package or have their own graphics and plotting capability. They also provide capability to I/O files and interface with other programming languages like C, C++, Fortran, etc ... Now a days some of them have parallel programming capabilities. I have not included MuPAD, a good symbolic math package, since is not really free. Check out if their most free license suits you.
Octave: An excellent package for numerical computations. It uses gnuplot for plotting and has a online help. It is also easily extensible (i.e. new functions, procedures can be written) either using its own language or by using dynamically loadable modules written in C, C++, Fortran or other languages. An extensive manual is available here. You can get a GNOME based front end for it here. It is distributed under the GNU Public License.
Scilab: Another superb package numerical computations having a good user interface and a very good online click-able help. Its plotting and graphic capabilities are also very impressive. It also provides for easy interfacing with Fortran and C. It has its own free license.
Yorick: Yorick is a fast, interpreted language, designed for scientific computing and numerical analysis. The syntax is similar to C, but the variables need not be declared. It offers an interactive graphics package based on X windows. X-Y plots, quadrilateral meshes, filled meshes, cell arrays, and contours are supported. You can embed compiled routines in Yorick to solve problems for which the interpreter is too slow. It is also useful as a pre and post processor for large physical simulation programs. A tutorial like manual is available here. Yorick is open source software, copyright of the Regents of the University of California.
Algae: As the above link describes it, Algae is a interpreted language for numerical analysis. It was developed as a fast and versatile tool, capable of handling large problems. Algae consists of the programming language Algae, and algae, the interpreter. Its features include speed (generally much faster than octave, RLaB and Scilab), storage of sparse arrays and a code profiling capability (to check where your code spends its time). A user manual is available here. It is distributed under the GNU General Public License.
YACAS: As the above link describes it, "YACAS is an easy to use, general purpose Computer Algebra System, a program for symbolic manipulation of mathematical expressions. It uses its own programming language designed for symbolic as well as arbitrary-precision numerical computations". Links to documentation (user manual, tutorial, etc ..) is available here. It is distributed under the GNU General Public License.
RLAB: The above link describes it thus, "Rlab is an interactive, interpreted scientific programming environment. Rlab is a very high level language intended to provide fast prototyping and program development, as well as easy data-visualization, and processing". It is distributed under the GNU General Public License. The author Ian Searle has written an article in The Linux Journal titled An Introduction to Rlab which as he reminds us, is a bit dated, and a Reference Manual is also available.
Maxima: Maxima is a symbolic computation program. The above link describes it as follows, "Maxima is a descendant of DOE Macsyma, which had its origins in the late 1960s at MIT. It is the only system based on that effort still publicly available and with an active user community, thanks to its open source nature. Macsyma was the first of a new breed of computer algebra systems, leading the way for programs such as Maple and Mathematica. This particular variant of Macsyma was maintained by William Schelter from 1982 until he passed away in 2001. In 1998 he obtained permission to release the source code under GPL".
The R-Project for Statistical Computing: R is a language and environment for statistical computing and graphics. It provides a large collection of tools for statistical analysis of large arrays of data and also graphical facilities. R is also a complete effective programming language. For computationally intensive tasks, C, C++ and Fortran code can be linked and called at run time. A comprehensive set of manuals dealing with installation, introduction, writing extensions, etc ... is available here. It is distributed under the GNU General Public License.
gTybalt: gTybalt is a step towards a free computer algebra system. It is object oriented, allowing symbolic calculations within C++. It is efficient, in the sense that solutions developed with gTybalt can be compiled with a C++ compiler and executed independently of gTybalt. The mathematical formulae are visualized using TeX fonts and can easily be converted to LaTeX. I did not realize that it has good graphic capabilities till I checked out the gTybalt manual. It is distributed under the GNU General Public License.
JACAL: As the link above describes it, " JACAL is an interactive symbolic mathematics program. JACAL can manipulate and simplify equations, scalars, vectors, and matrices of single and multiple valued algebraic expressions containing numbers, variables, radicals, and algebraic differential, and holonomic functions".
bc: bc is an arbitrary precision numeric processing language. It supports interactive execution of statements. Click here for a Manual in a variety of formats. It is GNU software and is distributed under the GNU General Public License.