- main - earth.li

Re: [ALUG] Recovering a NAS
by Chris Walker 14 Oct '16

14 Oct '16

On Thu, 29 Sep 2016 16:58:07 +0100 mick <mbm(a)rlogin.net> wrote: > On Tue, 27 Sep 2016 14:52:28 +0100 > Chris Walker <alug_cdw(a)the-walker-household.co.uk> allegedly wrote: > > [ some snips ] > > > > 2009 is a long time ago. Time to upgrade. :-) > > > > I didn't realise it was so long ago until I started looking for the > > receipt. It's stored as a scanned image on the NAS ;-) > > But of course you have a backup copy... :-) Isn't that what RAID is for? So, as you might well assume from the previous line, no I don't have another copy as I've always trusted RAID to deliver everything for me. I'm beginning to see the error of my ways though. > I'd agree with Steve. If you wish to continue to use a separate NAS, > then build a PC to hold the disks and copy the data to new disks. I've > had success with both debian and ubuntu in both raid 1 and 5. My > current NAS uses 4 * 2 TB disks in a raid 5 array (backed up to a 6 TB > USB disk) built using ubuntu server 16.04. (Incidentally, if you > haven't already seen it, the "Wintel Guy" has a useful site at > http://wintelguy.com/raidcalc.pl giving all sorts of useful info about > raid configurations.) Thanks. I'll take a look at that site. I'm tending to move to the 'build a PC for NAS' idea. I've looked at small(ish) cases and now need to start looking for suitable motherboards and power supplies. > > Noel Galer suggested that I look on ebay for a replacement Edimax > > NAS box. Been there, done that, but all the t-shirts were sold out. > > You don't want an old T shirt anyway. I've already got plenty :-) > > I think the drives hold something like 850GB of files of all sorts. > Given that, I'd go for 2 * 2 TB disks in raid 1. Ok. Assuming that's the route I take, and adopt your maxim of backing that up to a large USB disc, can I automate that? I don't mind copying stuff to a NAS but then don't want to have to go to the trouble of copying stuff to a USB disc as well. The reason I want it to be automatic is that I know things will be overlooked if they're not. P.S. I remembered to copy this to the list as well this time! -- __ __| |_ __ __ .----------------------------------------------. / _/ _` \ V V / | mailto:alug_cdw@the-walker-household.co.uk | \__\__,_|\_/\_/ |______________________________________________|

3 5

diff of binary files with helper program
by Mark Rogers 07 Oct '16

07 Oct '16

I need to perform diff against DBF (dBase database) files. dvview does a decent job of turning them into text files, but I don't want to have to convert all the files to text just to perform a diff. So I want a way to do something like: diff --helper=dbview file1.dbf file2.dbf .. or diff $(dbview file1.dbf) $(dbview file2.dbf) .. so that the files get converted to text on the fly. (Neither of those are remotely valid but hopely they get the idea across.) Is there a good way to do this? One objective is to do this using streams if possible, so that if file1.dbf and file2.dbf are very large there isn't any need to intermediate storage of files, but that isn't a huge problem if that's the only solution. -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) 21 Drakes Mews, Milton Keynes, MK8 0ER

1 1

2600 meeting at 18:00 on Friday the 7th October.
by Viperfang 04 Oct '16

04 Oct '16

Quick heads up about this months Norwich 2600 meeting, more info at http://www.norwich2600.co.uk

1 0

Re: [ALUG] Recording Streaming Audio
by Peter Berrie 02 Oct '16

02 Oct '16

Phil, Thanks a lot. Seems to be that if you stream at say 320k mp3, what will happen is that it will be decoded, imported into audacity, then when you save, it will be recoded into FLAC or WAV or whatever you choose. Probably the thing to do is take the FLAC version in the first place, then save to FLAC from Audacity, and there should be no losses. My service has an offline listening service as it happens, so you can use their app and download and then sign in, in some way, not sure whether that is on or off line, and play without re-streaming. If you leave the service all those saved tracks go. And having done this on one album, I cannot for the life of me find where they have stored the tracks. Today I was on the service, which is generally really nice, and got an album up. It turns out that all tracks cannot be either listened to online or downloaded, but have to be bought. This is basically a totally irritating business. I like to use my little Sandisk player sometimes when out, which is cheap and cheerful but has quite extraordinary sound quality for the price -- and more importantly, for the minimal weight and bulk. But the restrictions imposed by the streaming service with the aim of preserving copyright seem utterly ridiculous by comparison to the freedom you have with a bought CD, which you can rip and move around and listen to on whatever you want. I am now looking at mplayer, which lets you send the user agent string of your choice, as well as do user name and password. Probably there will be some ingenious and cunning way they have thought of to prevent that! Peter

1 0

Re: [ALUG] Recording Streaming Audio
by Phil Ashby 30 Sep '16

30 Sep '16

On 30/09/16 12:00, PB <peter.northerly(a)gmail.com> wrote: > Hi, > > Does anyone understand how Linux audio works, well enough to give me > some pointers on this? Hi Peter, I'll give it go, having spent far too long with my virtual arm up the insides of Linux audio stuff while getting radio stations working :) > ... > Sometimes I want to listen to the material on my tiny little portable > player when out and about. So I record the stream to file and copy > it across. I've discovered two fairly straightforward ways of doing > this. They are a bit manual, but I do not do it often so its OK. > > [recording the outgoing stream using gramofile or Audacity which > both produce WAV files - why?] This is simply because these tools pick up the raw, uncompressed digital audio stream going out of your SPDIF port, /after/ any app or media player has decoded the incoming (compressed) audio stream. They do this as it's guaranteed to work (no messing with decoders, or strange interfaces) and is always available either from the low level drivers (ALSA) or a preceding audio mixer such as PulseAudio. > There is also a package called streamripper, which seems like it > would do this, but I have not yet bothered. It lets you send a user > string corresponding to Firefox and to input user name and password. > And maybe mplayer will do it, and there is probably a way to input > user name and password on VLC from a command line startup. But the > above two methods work so I have not hacked around with alternatives > any further. You may have some success, it depends on how the streaming service protect their content using various Digital Rights Management (DRM) techniques (aka encryption/obfuscation). See below. > I would like to be able to save the stream in the mode in which its > received. If it is, for instance 320k mp3, then I would like to save > it as that, with no recoding which presumably will lower quality. > But something else must be going on with gramofile, because its WAV > that is written to disk. Similarly with Audacity. At what point are > they getting the stream and what are they doing with it? > ... > And if so, how do you get to write the digital stream being received, > with no conversions, just as it comes in, to file? @see above regarding where gramofile and Audacity get their audio from and why it's always WAV (raw) at this point. These raw audio recordings will be a bit-perfect copy of what you are hearing, no loss of quality. If you are really streaming MP3 files (not some weird AAC, WMA or other encrypted/protected format), then you /may/ be able to recover the file URL from your browser and use wget or curl to simply fetch the file. Most streaming services however go to great lengths to stop this as they don't want you taking a copy for yourself. Typical 'protection' mechanisms are: generating the majority of the browser content dynamically using obfuscated Javascript, embedding the media player in Adobe crud ware (Flashplayer), using custom codecs or whole applications that contain decryption keys, using proprietary protocols built on Websockets.. the list is endless and this is why tools like 'streamripper' may not work. I would continue using your existing technique(s), perhaps compressing the raw WAVs with flac or lame if you want to save disk space. Cheers, Phil. PS: It used to be possible to use a similar recording technique on Windows XP (from the 'Stereo Mix' device). Microsoft were encouraged by the recording industry to make that much more difficult in later versions, and some sound card manufacturers removed the capability entirely from their device drivers... think yourself lucky that won't happen on Linux :)

1 0

Recording Streaming Audio
by PB 30 Sep '16

30 Sep '16

Hi, Does anyone understand how Linux audio works, well enough to give me some pointers on this? I subscribe to an audio streaming service. I listen to this by accessing the material on Firefox, and this sends it over SPDIF out to an outboard combined DAC and headphone amp. So far everything is fine and it sounds great. However, this is at my desk....where we are not always sitting when wanting to listen to tunes. Sometimes I want to listen to the material on my tiny little portable player when out and about. So I record the stream to file and copy it across. I've discovered two fairly straightforward ways of doing this. They are a bit manual, but I do not do it often so its OK. One is a neat little command line tool called gramofile. You start it up in record mode, then tune in the stream, and it writes it to a WAV file of your choice. Why WAV, when its coming in as MP3? Dunno. The second is, you start pavucontrol, then Audacity, and after getting the source right, you just do record, and start up the stream. This records it, and you then save it as whatever you want. There is also a package called streamripper, which seems like it would do this, but I have not yet bothered. It lets you send a user string corresponding to Firefox and to input user name and password. And maybe mplayer will do it, and there is probably a way to input user name and password on VLC from a command line startup. But the above two methods work so I have not hacked around with alternatives any further. My puzzle is not about HOW to do it, its about WHAT this is doing. I would like to be able to save the stream in the mode in which its received. If it is, for instance 320k mp3, then I would like to save it as that, with no recoding which presumably will lower quality. But something else must be going on with gramofile, because its WAV that is written to disk. Similarly with Audacity. At what point are they getting the stream and what are they doing with it? When doing the above recording, what exactly is being recorded? Is it a digital stream that is being written to file? Is there any format conversion going on in gramofile or audacity? Are they maybe taking the analogue signal, and if so from where, and encoding it? And if so, how do you get to write the digital stream being received, with no conversions, just as it comes in, to file? PB

1 0

Disk to disk imaging
by Mark Rogers 28 Sep '16

28 Sep '16

I have a 2TB drive I want to copy to another 2TB drive. They're attached as USB devices to my Ubuntu desktop. I don't want to shut down and use a boot CD (eg Clonezilla) because I want to keep using my PC in the meantime. Assuming I want to copy from /dev/sdX to /dev/sdY, I know I can use dd if=/dev/sdX of=dev/sdY .. but it'll be slow because it'll copy loads of empty space. I know I can use partclone, which "understands" various filesystems and can skip empty space, but that won't clone the partitioning itself, nor I assume will it copy any boot sector stuff? What's the bit I'm missing, and/or is there a better way? Mark -- Mark Rogers // More Solutions Ltd (Peterborough Office) // 0844 251 1450 Registered in England (0456 0902) 21 Drakes Mews, Milton Keynes, MK8 0ER

3 5

Recovering a NAS
by Chris Walker 27 Sep '16

27 Sep '16

In 2009 I bought an Edimax NAS from Scan.co.uk and it's worked fine since, albeit with a change of hard drives. However over the weekend it wouldn't power up cleanly. Neither of the hard drive lights would come on and it wouldn't come ready. I also couldn't communicate with it in any way. I got in touch with Edimax and they suggested doing a reset on it. That at least allowed me to communicate with it and it came ready, but it said that the discs weren't installed correctly although it did show the partitions. I removed both drives (WD 1TB Red drives in RAID1) and using mdadm, I can see the contents of disc 1 and it all looks ok. If I buy another NAS box, is there any way I can simply put these drives in it and expect it to work, or should I expect the new box to have a different arrangement somehow of formatting? What I'm seeking to avoid is loads of copying and moving of files. Are there any recommendations from people here as to a small (I don't want huge amounts of storage and I power off the box when not used) NAS? -- __ __| |_ __ __ .----------------------------------------------. / _/ _` \ V V / | mailto:alug_cdw@the-walker-household.co.uk | \__\__,_|\_/\_/ |______________________________________________|

3 2

Re: [ALUG] Recovering a NAS
by Noel Galer 27 Sep '16

27 Sep '16

>-----Original Message----- >From: main [mailto:main-bounces@lists.alug.org.uk] On Behalf Of Chris >Walker >Sent: Tuesday, September 27, 2016 10:10 AM >To: main(a)lists.alug.org.uk >Subject: [ALUG] Recovering a NAS > >In 2009 I bought an Edimax NAS from Scan.co.uk and it's worked fine >since, albeit with a change of hard drives. > >However over the weekend it wouldn't power up cleanly. Neither of the >hard drive lights would come on and it wouldn't come ready. I also >couldn't communicate with it in any way. > >I got in touch with Edimax and they suggested doing a reset on it. That >at least allowed me to communicate with it and it came ready, but it >said that the discs weren't installed correctly although it did show >the partitions. > >I removed both drives (WD 1TB Red drives in RAID1) and using mdadm, I >can see the contents of disc 1 and it all looks ok. > >If I buy another NAS box, is there any way I can simply put these >drives in it and expect it to work, or should I expect the new box to >have a different arrangement somehow of formatting? > >What I'm seeking to avoid is loads of copying and moving of files. > >Are there any recommendations from people here as to a small (I don't >want huge amounts of storage and I power off the box when not used) NAS? > >-- > __ __| |_ __ __ .----------------------------------------------. >/ _/ _` \ V V / | mailto:alug_cdw@the-walker-household.co.uk | >\__\__,_|\_/\_/ |______________________________________________| > > I think as you suggest, any other NAS enclosure would likely have a different setup and the disks would not just transfer across. It is my understanding that much of a RAID drives configuration is actually on the drive itself with only some of it in the controller. Personally I would look on eBay and try to locate an identical unit which I would expect to 'just work'. However, if the worst comes to the worst and you are forced to copy huge tranches of files and are looking for a way to feel sure that the source and destination might actually match, plus you can put up with using Windows for a while ( assuming your disk formats are compatible ), I would thoroughly recommend trying a fabulous tool that must have been designed with us in mind. It is called 'Beyond Compare' by Scooter Software and I have been using it every day for over a decade. You can easily see what is on the source and destination disks side by side and copy all or part of you disk, at once or in recoverable stages and even compare multiple files and folders byte by byte to ensure all is well. See here :- http://www.scootersoftware.com/ Good luck, Noel ( Xorsyst ) _______________________________________________ main(a)lists.alug.org.uk http://www.alug.org.uk/ https://lists.alug.org.uk/mailman/listinfo/main Unsubscribe? See message headers or the web site above!

2 1

Transferring servers
by steve-ALUG＠hst.me.uk 20 Sep '16

20 Sep '16

Hi, My in-home server is at the end of it's tether, and I need to move to new hardware (motherboard, psu, case, fans etc) before the old one finally fails. This is a very stupid question. Under windows, in days of old (win 95, 98, xp for instance), I've had some success in the past, with just transferring hard disks into the new machine, and then installing relevant drivers. Would this work under linux? (I'm running xubuntu) If not, what's the best way to go? Installing the same version of xubuntu on the new machine with the same packages as the old one, then copying the data and config files, or copying everything from the old machine and then booting from a live cd and installing from there over the top of the transferred data, then tweaking or restoring the customised config files, or something else? I'm currently raided (Raid 1, 2 disks mirrored). Presumably I should do this again? I have a / and a home partition. Again, should I stick with this? Should I be using LVM (Logical Volume Management)? If so, how can I do it with RAID? Any/all advice gratefully received! Steve

5 7