Skip to content

Commit 78cdb21

Browse files
committed
Wrote introduction, readme.rst now points to the introduction page to keep things consistent
1 parent 65dacbb commit 78cdb21

4 files changed

Lines changed: 128 additions & 51 deletions

File tree

README.rst

Lines changed: 0 additions & 51 deletions
This file was deleted.

README.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
doc/source/intro.rst

doc/source/api.rst

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
.. _api-label:
2+
3+
#############
4+
API Reference
5+
#############
6+
7+
****************
8+
smmap.mman
9+
****************
10+
11+
.. automodule:: smmap.mman
12+
:members:
13+
:undoc-members:
14+
15+
****************
16+
smmap.buf
17+
****************
18+
19+
.. automodule:: smmap.buf
20+
:members:
21+
:undoc-members:
22+
23+
****************
24+
smmap.exc
25+
****************
26+
27+
.. automodule:: smmap.exc
28+
:members:
29+
:undoc-members:
30+
31+
****************
32+
smmap.util
33+
****************
34+
35+
.. automodule:: smmap.util
36+
:members:
37+
:undoc-members:
38+
39+
40+
41+
42+

doc/source/index.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,9 @@ Contents:
1212
.. toctree::
1313
:maxdepth: 2
1414

15+
intro
16+
tutorial
17+
api
1518
changes
1619

1720
Indices and tables

doc/source/intro.rst

Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
###########
2+
Motivation
3+
###########
4+
When reading from many possibly large files in a fashion similar to random access, it is usually the fastest and most efficient to use memory maps.
5+
6+
Although memory maps have many advantages, they represent a very limited system resource as every map uses one file descriptor, whose amount is limited per process. On 32 bit systems, the amount of memory you can have mapped at a time is naturally limited to theoretical 4GB of memory, which may not be enough for some applications.
7+
8+
########
9+
Overview
10+
########
11+
12+
Smmap wraps an interface around mmap and tracks the mapped files as well as the amount of clients who use it. If the system runs out of resources, or if a memory limit is reached, it will automatically unload unused maps to allow continued operation.
13+
14+
To allow processing large files even on 32 bit systems, it allows only portions of the file to be mapped. Once the user reads beyond the mapped region, smmap will automatically map the next required region, unloading unused regions using a LRU algorithm.
15+
16+
The interface also works around the missing offset parameter in python implementations up to python 2.5.
17+
18+
Although the library can be used most efficiently with its native interface, a Buffer implementation is provided to hide these details behind a simple string-like interface.
19+
20+
For performance critical 64 bit applications, a simplified version of memory mapping is provided which always maps the whole file, but still provides the benefit of unloading unused mappings on demand.
21+
22+
#############
23+
Prerequisites
24+
#############
25+
* Python 2.4, 2.5 or 2.6
26+
* OSX, Windows or Linux
27+
28+
The package was tested on all of the previously mentioned configurations.
29+
30+
###########
31+
Limitations
32+
###########
33+
* The memory access is read-only by design.
34+
* In python below 2.6, memory maps will be created in compatibility mode which works, but creates inefficient memory mappings as they always start at offset 0.
35+
* It wasn't tested on python 2.7 and 3.x.
36+
37+
###############
38+
Getting Started
39+
###############
40+
It is advised to have a look at the :ref:`Usage Guide <tutorial-label>` for a brief introduction on the different database implementations.
41+
42+
################
43+
Installing smmap
44+
################
45+
Its easiest to install smmap using the *easy_install* or *pip* program, which is part of the `setuptools`_ or `pip`_ respectively::
46+
47+
$ easy_install smmap
48+
# or
49+
$ pip install smmap
50+
51+
As the command will install smmap in your respective python distribution, you will most likely need root permissions to authorize the required changes.
52+
53+
If you have downloaded the source archive, the package can be installed by running the ``setup.py`` script::
54+
55+
$ python setup.py install
56+
57+
##################
58+
Homepage and Links
59+
##################
60+
The project is home on github at `https://github.com/Byron/smmap <https://github.com/Byron/smmap>`_.
61+
62+
The latest source can be cloned from github as well:
63+
64+
* git://github.com/gitpython-developers/smmap.git
65+
66+
67+
For support, please use the git-python mailing list:
68+
69+
* http://groups.google.com/group/git-python
70+
71+
72+
Issues can be filed on github:
73+
74+
* https://github.com/Byron/smmap/issues
75+
76+
###################
77+
License Information
78+
###################
79+
*smmap* is licensed under the New BSD License.
80+
81+
.. _setuptools: http://peak.telecommunity.com/DevCenter/setuptools
82+
.. _pip: http://www.pip-installer.org/en/latest/

0 commit comments

Comments
 (0)