AminetAminet
Search:
82753 packages online
About
Recent
Browse
Search
Upload
Setup
Services

comm/tcp/S3C.lha

Mirror:Random
Showing:m68k-amigaosppc-amigaosppc-morphosi386-arosi386-amithlonppc-warpupppc-powerupgeneric
No screenshot available
Short:Backup,restore+manage files on Amazon S3
Author:Michal Ludvig / Alex Carmona
Uploader:Alex Carmona <AmigaOne theworld net>
Type:comm/tcp
Version:1.1
Architecture:ppc-amigaos >= 4.0.0
URL:http://s3tools.org
Date:2013-02-07
Download:http://aminet.net/comm/tcp/S3C.lha - View contents
Readme:http://aminet.net/comm/tcp/S3C.readme
Downloads:373
OVERVIEW:

S3Cmd is a Python script and libraries which give you access to Amazon S3
from the Shell. It also provides support for CloudFront and S3 Websites.

Most of the online storage and backup services out there use proprietary
Windows/OSX clients or require using a browser to upload data.
As Amazon provide an API to connect directly to their servers I thought that
it would be nice if the Amiga too could have transparent access to their cloud
storage service.

After exploring the various solutions I settled for S3Cmd as it only needs
Python which comes pre-installed with OS4 and I wanted to keep it as simple to
install as possible so that anyone else could use it too.

With the included S3Cmd script you will be able to easily load/save, share and
backup/restore data on the cloud with your Amiga.
No more excuses for not having backups ;-)


REQUIREMENTS:

For S3Cmd to be of any use you'll need an Amazon Web Services (AWS) account.

Currently Amazon gives a whole year for free from the date you sign up and their
free offer includes a lot more than storage. For example you can create an M1
cloud server instance with 30GB of EBS storage, dedicated IP, root access and
the ability to install any Linux version and software you want.
See http://aws.amazon.com/free/ for details


INSTALLATION:

1) Copy the S3C drawer wherever you like

2) If you don't have a $HOME variable defined (try 'GetEnv HOME' to find out),
open the Shell and type:

'MakeDir ENVARC:HOME'

'SetEnv SAVE HOME ENVARC:HOME'

You can use any drawer you like instead of ENVARC:HOME, it just seems like the
best place for it as $HOME is often used by Linux programs to store settings.

3) In the Shell, type: 'DH1:S3C/S3Cmd --configure' (replace DH1: with your path)

It will ask for your AWS keys and whether you want to use GPG, HTTPS and
proxies.

- If you have GPG installed you could try it otherwise answer 'n' and press
Enter   when it asks for the GPG passphrase.

- I recommend using HTTPS just to be sure your keys aren't sent as plain text.

- I haven't tested the GPG and Proxy support so I don't know if they work or
not.

- If needed you can reconfigure S3Cmd later with '--configure'.


USAGE:

After installing S3Cmd you will be able to access your S3 storage transparently
from the Shell with the S3Cmd command. You don't have to 'CD' to the S3C drawer
first, you can call it from anywhere.

You can use S3Cmd as-is (see the How-To in the S3C/README and 'S3Cmd --help'),
for example:

'S3Cmd mb s3://amigarulez' (Create a bucket named "amigarulez" if available)

'S3Cmd put DH1:myfile s3://amigarulez/newdrawer/' (put a file in a new drawer)

'S3Cmd ls s3://amigarulez/newdrawer/' (list contents of newdrawer)

'S3Cmd get s3://amigarulez/newdrawer/myfile RAM:myfile' (download file to RAM:)

'S3Cmd sync DH1:MyPrecious/ s3://amigarulez/MyPrecious/' (incremental backup)

'S3Cmd sync s3://amigarulez/MyPrecious/ DH1:MyPrecious/' (incremental restore)

'S3Cmd -r del s3://amigarulez/newdrawer' (delete newdrawer recursively)

'S3Cmd rb s3://amigarulez' (remove the amigarulez bucket)

or you can create aliases and scripts to have more amiga-like commands operating
on a single bucket for common tasks such as:

S3List, S3Put, S3Get, S3Delete, S3Info, etc.
For example, if you own a bucket called "amigarulez":

'Alias S3       "DH1:S3C/S3Cmd"'
'Alias S3List   S3 s3://amigarulez/[]'
'Alias S3Put    S3 -r [] s3://amigarulez/'
'Alias S3Get    S3 -r s3://amigarulez/[]'
'Alias S3Delete S3 del s3://amigarulez/[]'
'Alias S3DelAll S3 -r del s3://amigarulez/[]'
'Alias S3Info   S3 info s3://amigarulez/[]'

More complex operations like copying files to a new drawer or moving/renaming
objects require multiple arguments which can't be concatenated with Alias so
you would have to create small scripts, for example an S:Shell/S3Move script:

.key SOURCE/A,DESTINATION/A
DH1:S3C/S3Cmd mv s3://amigarulez/<SOURCE> s3://amigarulez/<DESTINATION>

or a more flexible S3Put script than the S3Put alias above, S:Shell/S3Put:

.key SOURCE/A/M,DESTINATION/A
DH1:S3C/S3Cmd put -r <SOURCE> s3://amigarulez/<DESTINATION>

or an S3Copy script to duplicate files on the remote side, S:Shell/S3Copy:

.key SOURCE/A,DESTINATION/A
DH1:S3C/S3Cmd cp s3://amigarulez/<SOURCE> s3://amigarulez/<DESTINATION>

Finally, a script to automatically make either daily, weekly, or monthly backups
of the SYS: partition, incrementally (it's fast as it only copies newer file).
(To be useful it should be placed in WBStartup or loaded by some cron job)

Lab Top
Wait UNTIL 12:00 ; set a time when the machine will be ON with light use
If `Date LFormat %e` EQ 1 ; 1st of the month
 DH1:S3C/S3Cmd --delete-removed sync SYS: s3://amigarulez/backups/monthly/
Else If `Date LFORMAT %w` EQ 1 ; Monday
 DH1:S3C/S3Cmd --delete-removed sync SYS: s3://amigarulez/backups/weekly/
Else ; daily
 DH1:S3C/S3Cmd --delete-removed sync SYS: s3://amigarulez/backups/daily/
EndIf
Skip Top BACK ; Jump to Top and wait again (most likely until next day)


HINTS:

Trailing slashes matters, always append a slash "/" after drawer names unless
you are trying to delete or get some information on the drawer itself.

With 'put' and 'sync' if you specify the source as "RAM:Drawer" instead of
"RAM:Drawer/" it will create two drawers "RAM" and "RAM/Drawer" which probably
isn't what you want.

I recommend using a single bucket and neatly organized drawers as it's easier
to have a constant S3 URI for your scripts and aliases.

If you want to multiple buckets, you could create an ENV:S3Bucket variable and
then put $S3Bucket instead of the bucket name in your scripts, that way you
can use 'SetEnv S3Bucket name' to switch buckets without modifying the scripts.
With aliases you can't store a reference to a variable so you'd have to use
escaped backticks like this: 'Alias S3List S3 ls s3://*`GetEnv S3Bucket*`/[]' 

If you enable user read permission on drawers or files (setacl) you can share
your files by giving people links such as:
http://s3.amazonaws.com/amigarulez/photos/myphoto.jpg or
http://amigarulez.s3.amazonaws.com/photos/myphoto.jpg

Use the 'S3Cmd du' command to keep an eye on how much storage you use, remember
that you can only use up to 5GB for free on S3. As you also get plenty of
storage on the EC2/EBS side, you could always move stuff from S3 to EBS.

If you have Python: on an FFS partition, you can speed it up a great deal by
activating caching with 'C:fs_plugin_cache SYS: S=20000' (20MB cache).

With ARexx for Python and ProAction from Andy Broad you can quickly create some
ReAction GUIs to interact with S3. 

If someone has the time and patience to make an s3-handler to mount buckets as
volumes like FTPMount and NetFS, please do!

If you run into issues, ask on the AW forum, that's the one I am most likely
to visit regularly.


CREDITS:

All the hard work was done by Michal Ludvig (S3Cmd), I only added a few lines
of code to prevent infinite loops and crashes from the colon in volume names.


Enjoy!


Contents of comm/tcp/S3C.lha
PERMISSION  UID  GID    PACKED    SIZE  RATIO METHOD CRC     STAMP     NAME
---------- ----------- ------- ------- ------ ---------- ------------ ----------
[unknown]                 4399    5052  87.1% -lh5- 7f86 Feb  5 06:00 S3C.info
[unknown]                 1263    2707  46.7% -lh5- 300f Jul 10  2012 s3c/install
[unknown]                 6924    9486  73.0% -lh5- e10a Feb  6 21:21 S3C/INSTALL.info
[unknown]                 3381    7651  44.2% -lh5- e101 Jul 10  2012 s3c/news
[unknown]                 6924    9486  73.0% -lh5- 230e Feb  6 21:21 S3C/NEWS.info
[unknown]                  367     661  55.5% -lh5- 63bf Jul 10  2012 s3c/pkg-info
[unknown]                 6924    9486  73.0% -lh5- 4f34 Feb  6 21:21 S3C/PKG-INFO.info
[unknown]                 4850   13130  36.9% -lh5- c80a Jul 10  2012 s3c/readme
[unknown]                 6699    9486  70.6% -lh5- 1d74 Feb  6 21:23 S3C/README.info
[unknown]                 4398    5052  87.1% -lh5- 64b5 Feb  6 19:01 S3C/S3.info
[unknown]                 1048    2970  35.3% -lh5- a95b Jul 10  2012 S3C/S3/AccessLog.py
[unknown]                 1537    3621  42.4% -lh5- 2f0b Feb  4 04:13 S3C/S3/AccessLog.pyc
[unknown]                 1851    7065  26.2% -lh5- b161 Jul 10  2012 S3C/S3/ACL.py
[unknown]                 3063    8307  36.9% -lh5- db9f Feb  4 04:13 S3C/S3/ACL.pyc
[unknown]                  418    1069  39.1% -lh5- d165 Jul 10  2012 S3C/S3/BidirMap.py
[unknown]                  728    1948  37.4% -lh5- 20dd Feb  4 04:13 S3C/S3/BidirMap.pyc
[unknown]                 6818   31433  21.7% -lh5- cc15 Jul 10  2012 S3C/S3/CloudFront.py
[unknown]                 9536   26087  36.6% -lh5- 2c50 Feb  4 04:13 S3C/S3/CloudFront.pyc
[unknown]                 2614    7654  34.2% -lh5- 2891 Jul 10  2012 S3C/S3/Config.py
[unknown]                 3536    7983  44.3% -lh5- 87b4 Feb  4 04:13 S3C/S3/Config.pyc
[unknown]                  914    2664  34.3% -lh5- de31 Jul 10  2012 S3C/S3/Exceptions.py
[unknown]                 1500    4148  36.2% -lh5- bb0a Feb  4 04:13 S3C/S3/Exceptions.pyc
[unknown]                 4102   14366  28.6% -lh5- bba4 Jul 10  2012 S3C/S3/FileLists.py
[unknown]                 4515    9486  47.6% -lh5- 87d7 Feb  4 04:13 S3C/S3/FileLists.pyc
[unknown]                 1511    4721  32.0% -lh5- 6193 Jul 10  2012 S3C/S3/MultiPart.py
[unknown]                 2242    4991  44.9% -lh5- 408c Feb  4 04:13 S3C/S3/MultiPart.pyc
[unknown]                  296     473  62.6% -lh5- bbd2 Jul 10  2012 S3C/S3/PkgInfo.py
[unknown]                  416     635  65.5% -lh5- bec1 Feb  4 04:13 S3C/S3/PkgInfo.pyc
[unknown]                 1432    6017  23.8% -lh5- d293 Jul 10  2012 S3C/S3/Progress.py
[unknown]                 2180    5499  39.6% -lh5- 16da Feb  4 04:13 S3C/S3/Progress.pyc
[unknown]                 8570   37842  22.6% -lh5- b6d7 Jul 10  2012 S3C/S3/S3.py
[unknown]                11875   29085  40.8% -lh5- 769b Feb  4 04:13 S3C/S3/S3.pyc
[unknown]                 1651    6560  25.2% -lh5- b9f7 Jul 10  2012 S3C/S3/S3Uri.py
[unknown]                 2840    9111  31.2% -lh5- 99f5 Feb  4 04:13 S3C/S3/S3Uri.pyc
[unknown]                 1877    6903  27.2% -lh5- 76e6 Jul 10  2012 S3C/S3/SimpleDB.py
[unknown]                  753    1943  38.8% -lh5- 9ba5 Jul 10  2012 S3C/S3/SortedDict.py
[unknown]                 1203    2574  46.7% -lh5- 8826 Feb  4 04:13 S3C/S3/SortedDict.pyc
[unknown]                 3933   12534  31.4% -lh5- f32c Feb  6 19:34 S3C/S3/Utils.py
[unknown]                 5633   13648  41.3% -lh5- 8e25 Feb  6 19:35 S3C/S3/Utils.pyc
[unknown]                  106     121  87.6% -lh5- d285 Feb  4 04:13 S3C/S3/__init__.pyc
[unknown]                 3282    7256  45.2% -lh5- df59 Feb  7 02:02 S3C/S3C.readme
[unknown]                 4118    4908  83.9% -lh5- 0f63 Feb  6 21:02 S3C/S3C.readme.info
[unknown]                20808   83206  25.0% -lh5- 58c9 Feb  4 04:07 S3C/S3Cmd
[unknown]                 9917   15536  63.8% -lh5- cd5e Feb  6 21:20 S3C/S3Cmd.info
[unknown]                 5060   13655  37.1% -lh5- 4676 Jul 10  2012 S3C/s3cmd.man
[unknown]                 7045    9546  73.8% -lh5- 68ed Feb  6 21:20 S3C/s3cmd.man.info
---------- ----------- ------- ------- ------ ---------- ------------ ----------
 Total        46 files  185057  477762  38.7%            Feb  7 20:30

Aminet © 1992-2019 Urban Müller and the Aminet team. Aminet contact address: <aminetaminet net>