re: Problems With the Number of Images on the Stamporama Server - Discussion
I agree with Kapul. Why six months? That does seem like a long time. I always kept a scan of the books myself and when the book was gone, so was the scan.
Wonder how many of us really knew the scans were kept so long?
I vote to shorten to three months.
2 Members like this post. Login to Like.
"Appreciate the beauty of nature. You never know when life will deal you an unplanned twist of fate. Enjoy things while you can, because you just never know."
re: Problems With the Number of Images on the Stamporama Server - Discussion
You are all quite right that the amount of time that we need to keep the Auction/Approval images is negotiable. First up, I'm working on getting them back to 6 months. Once I achieve that and we see how that goes we'll talk about bringing the retention time back down to 4 or 3 months.
re: Problems With the Number of Images on the Stamporama Server - Discussion
I would not go below four months on the image retention, while considering that six months might be best. This is because international mail could take four to eight weeks depending on where its going. In the case of a problem with the items shipped, members will need to be able to use the images to verify if there is a problem. Add time for a member to buy additional items and then invoicing, and this could get up to 90 days, provided there is no return shipment involved as well. Then there is the time needed for the Auctioneer if the parties to the transaction can't resolve the matter, and one files a dispute.
re: Problems With the Number of Images on the Stamporama Server - Discussion
Okay, my knee-jerk reaction question would be ... at what point does it become the sellers responsibility to help in resolving a situation by keeping their own scan of the sheet/page they have up for sale?
Just thinking some random thoughts that it sounds like the sellers have been depending on the staff to do way too much work. Is this how it works in a real life club activity? I don't belong to a real life club and have no experience with dealing that way. I thought it was like other purchases where the buyer takes the opportunity to purchase something, pays for it right away and if not satisfied then deals with the seller directly.
Once again, this is my random thoughts at 4:20AM. Maybe I'm over thinking this....
1 Member likes this post. Login to Like.
"Appreciate the beauty of nature. You never know when life will deal you an unplanned twist of fate. Enjoy things while you can, because you just never know."
re: Problems With the Number of Images on the Stamporama Server - Discussion
Not sure how others work but I keep my scan of sale items on my computer in original form and resized for StampoRama. Think some are even duplicated on my laptop and a memory stick.
However I can see that 6 months is a reasonable hold time from a dispute viewpoint.
re: Problems With the Number of Images on the Stamporama Server - Discussion
I noticed last evening that most of the USA listings posted by jameskaczo have no images showing (Image Not Available).
Is this a result of images being stored on Stamporama for auction lots listed over 6 months ago that were never sold?
Another question - I try to delete ALL of my unsold lots that are older than 3 months. When I delete unsold lots are the images automatically removed from the system?
re: Problems With the Number of Images on the Stamporama Server - Discussion
If there was a way of selecting muliple images for bulk deletion regarding sold lots that have been received by the buyer, I would gladly delete my own images from the system, but trying to delete images one by one is just too time consuming when one is listing many lots in the auction or in the Approval Book sections.
BuckaCover.com - 80,000 covers priced 60c to $1.50 - Easy browsing 300 categories 25 Oct 2017 09:19:43am
re: Problems With the Number of Images on the Stamporama Server - Discussion
"I would gladly delete my own images from the system,"
That is a caring offer, however designing a system that relies on voluntary compliance with policies is a waste of effort on the part of the site designer. There will always be those, like you (and many other active members) who would do anything they could on a voluntary basis to improve the use of a site such as this. If all members were like this, voluntary policies could work.
Unfortunately, there will always be those that use any system to their own exclusive advantage, without regard to the greater good. The role of the site designer and the management team is to protect the greater good from the actions of those few. That is the function of "policies" (which some think of as "rules").
So, to all members, I say, when you next encounter a "rule change" think not of how it is affecting you, because "I am a reasonable person and this is affecting my freedom of action". Think instead of the purpose of the policy, because it will invariably be aimed at curtailing the actions of a very few who take advantage of a loophole, or just don't care about the greater good. It is unfortunate, but necessary, that the policy also affects the behavior of the reasonable person who would not otherwise abuse whatever situation is being addressed.
None of this has anything to do with the number of images on the system. That is just a technical issue. Patches' comment just gave me an opportunity to "get on my soapbox" about something that is always on my mind.
Roy
2 Members like this post. Login to Like.
"BuckaCover.com - 80,000 covers priced 60c to $1.50- 10,000+ new covers coming Tuesday June 1"
BuckaCover.com - 80,000 covers priced 60c to $1.50 - Easy browsing 300 categories 25 Oct 2017 10:23:08am
re: Problems With the Number of Images on the Stamporama Server - Discussion
"at what point does it become the sellers responsibility to help in resolving a situation by keeping their own scan of the sheet/page they have up for sale?"
This is an excellent point.
It is ALWAYS the buyers' and sellers' ultimate responsibility to "help in resolving a situation". It is important to remember that this site is an entirely volunteer effort. Nobody is being paid, hosting of the site is being donated.
By the nature of their roles and willingness to donate their time, volunteers are operating on a "best efforts" basis, and cannot be held responsible for an inability to solve a situation.
Roy
Login to Like this post
"BuckaCover.com - 80,000 covers priced 60c to $1.50- 10,000+ new covers coming Tuesday June 1"
re: Problems With the Number of Images on the Stamporama Server - Discussion
Liz, sometimes someone's bulk upload file will have a tiny error in it, and it can blow out the data causing problems in the auction lots such as images missing. When we find this, we ask the seller to check for problems and correct them. Thanks for mentioning it.
re: Problems With the Number of Images on the Stamporama Server - Discussion
There is a way to manually delete one's images that were uploaded to the site server. However, the problem with doing that is that one might inadvertently delete images that are part of the various conversations in the Discussion Board. Deletion of those images could render a discussion thread useless.
re: Problems With the Number of Images on the Stamporama Server - Discussion
Just a quick update on the image file clean up process. I have done a fair amount of analysis on the images, where they came from (Auctions, Approvals etc) and have a plan in place. My initial target is to reduce the number of images on the site by 39GB, after which it will take more analysis to break it down further. I have a program in place that is running every night removing images along with old closed Auction Lots that have been closed in excess of 6 months. That program has so far reduced the images from 74GB to 66GB. As you can see I have a lot of work to do.
re: Problems With the Number of Images on the Stamporama Server - Discussion
So that people understand a small catch to the image deletion routine, if you are using the same images from old auction lots that haven't sold in over six months, those images are being deleted as well, even if the old auction lots are being relisted over and over again non-stop for more than 6 months. One would sort of think that such relisting is an exercise in futility...
Silence in the face of adversity is the father of complicity and collusion, the first cousins of conspiracy.. 28 Oct 2017 08:05:17pm
re: Problems With the Number of Images on the Stamporama Server - Discussion
" .... Seems to me a pair of 240gb SSDs one for primary storage and one for backup would speed things up precipitously.
The only problem is that they don't grow on trees ($90 or so apiece). ...."
Is there a simple way to solve the problem like what ever an SSD is ?
Is that $90,00 a monthly fee, or a flat price ?
Would there be other expenses involved ?
Is there a similar device in the tetra bite range "
This is all a puzzlement to me.
Login to Like this post
".... You may think you understood what you thought I said, but I'm not sure you realize that what you think you heard is not what I thought I meant. .... "
Silence in the face of adversity is the father of complicity and collusion, the first cousins of conspiracy.. 28 Oct 2017 08:27:50pm
re: Problems With the Number of Images on the Stamporama Server - Discussion
" .... I noticed last evening that most of the USA listings posted by (A member) have no images showing (Image Not Available).
Is this a result of images being stored on Stamporama for auction lots listed over 6 months ago that were never sold? ...."
Six months and un-sold ?
I will go out on a limb and opine that that is something in the nature of taking advantage of the clubs system. I noticed those posts of what I consider slightly overpriced discount postage sheets clogging the auction many times with few sales and apparently zero participation in other club activities.
Login to Like this post
".... You may think you understood what you thought I said, but I'm not sure you realize that what you think you heard is not what I thought I meant. .... "
BuckaCover.com - 80,000 covers priced 60c to $1.50 - Easy browsing 300 categories 29 Oct 2017 01:52:33pm
re: Problems With the Number of Images on the Stamporama Server - Discussion
"Seems to me a pair of 240gb SSDs one for primary storage and one for backup would speed things up precipitously. "
It's not as easy as that.
Stamporama is hosted on a leased dedicated server in a "server farm". "Dedicated" means that it is all mine, nobody anonymous shares it (as opposed to "shared hosting"). Only my websites, a very few client websites and Stamporama are on this server.
Storage is provided by 3 x 2TB (2 terabytes = 2,000 GB) drives in a hardware RAID configuration. That means they back themselves up and if one fails, the data is all intact on the other two and the failed drive can be "hot-swapped" (replaced without shut-down) without disruption.
The slowdown is not due to local storage, it is due to an off-site backup. I had a server crash early this year (luckily it was a secondary server that I was in the process of replacing, but a few clients were still on it.) It had a main drive and a backup drive as Antonio suggests. Basically, the server "fried" and took BOTH drives down with it. Lost everything except what I had on my own desktop. No problem restoring the websites themselves, but all the uploaded images etc were gone permanently. Clients had to restore those from their own archives.
That convinced me to engage an offsite backup with this server. I have 1TB offsite backup. The backup files are transmitted to an offsite server via FTP. The time involved is the time to create the massive backup file and transmit it.
The backup processing time is already down by an hour, and we have only trimmed the upload folder by 10GB. Another estimated 30GB to go!
Roy
6 Members like this post. Login to Like.
"BuckaCover.com - 80,000 covers priced 60c to $1.50- 10,000+ new covers coming Tuesday June 1"
re: Problems With the Number of Images on the Stamporama Server - Discussion
I just discovered that all of my thumbnails for uploaded images 31 through 209 are gone! Numbers 1 through 30 are still being displayed. This makes it far more difficult to find previously uploaded images in order to reuse them. Can this be fixed?
Login to Like this post
"I no longer collect, but will never abandon the hobby"
Dialysis, damned if you do...dead if you don't 01 Nov 2017 11:46:42am
re: Problems With the Number of Images on the Stamporama Server - Discussion
Roy,
I have always found offsite backups as problematic; the performance impact alone was always enough to cause me to pull my hair out.
I moved all my RAID servers over to external drive backups. I used RAID controllers with external connectors and then simply rotated the external drives each day. One of the huge advantages of this is that I could do a full file system copy instead of using lame ‘compressed’ backup software. This made is extremely easy and cost efficient to fetch a lost file or even restore an entire OS of drive without needing to own (and install after a crash) expensive backup software. Without a high end RAID controller and fast drives the full file system copies are fast, I simply wrote some batch files and used the OS scheduler to run them every night. I established external 7 drives and then each morning I would swap them. Then every month I would rotate one of the external drives out of the weekly 7 (replacing it) and keep it offsite. I have managed some very large rack servers farms this way and would never go back to any other method.
Drives are cheap. The situation you have now, where you will need to keep deleting files every few months to maintain your cloud storage solution seems a significant restriction which will only get worse (it this site and your hosting is growing).
Don
BuckaCover.com - 80,000 covers priced 60c to $1.50 - Easy browsing 300 categories 01 Nov 2017 03:05:40pm
re: Problems With the Number of Images on the Stamporama Server - Discussion
"I just discovered that all of my thumbnails for uploaded images 31 through 209 are gone! Numbers 1 through 30 are still being displayed. This makes it far more difficult to find previously uploaded images in order to reuse them. Can this be fixed?"
If they have been wiped, they were more than 6 months old. Afraid they are gone.
I am afraid the image numbers don't mean anything to me. It is probably just a consecutive numbering of your auction lots as they are pulled out of the database.
Roy
Login to Like this post
"BuckaCover.com - 80,000 covers priced 60c to $1.50- 10,000+ new covers coming Tuesday June 1"
BuckaCover.com - 80,000 covers priced 60c to $1.50 - Easy browsing 300 categories 01 Nov 2017 03:10:30pm
re: Problems With the Number of Images on the Stamporama Server - Discussion
" I established external 7 drives and then each morning I would swap them. Then every month I would rotate one of the external drives out of the weekly 7 (replacing it) and keep it offsite. "
That's all fine if you have physical access to the server (which also implies wide bandwidth access to the internet, i.e. 1 Gbps), however that's not possible for us. The server is in a server farm and that sort of physical management, and the leasing of a server with that many extra drives, would cost a fortune.
Roy
Login to Like this post
"BuckaCover.com - 80,000 covers priced 60c to $1.50- 10,000+ new covers coming Tuesday June 1"
re: Problems With the Number of Images on the Stamporama Server - Discussion
Roy said:
"If they have been wiped, they were more than 6 months old. Afraid they are gone."
I don't know if all were more than six months old, but most certainly were.
I guess I misunderstood. I thought the image cleanup was limited to images associated with auctions and sales books. All of my missing thumbnails are related to my forum posts.
Tom
Login to Like this post
"I no longer collect, but will never abandon the hobby"
re: Problems With the Number of Images on the Stamporama Server - Discussion
Roy,
Stamporama runs on Linux? Please look into using rsync instead of FTP for backups, FTP is kind of obsolete and insecure - plus rsync you can use to only backup what has changed, and supports compression and encryption (via ssh).
re: Problems With the Number of Images on the Stamporama Server - Discussion
Roy said:
"I am afraid the image numbers don't mean anything to me. It is probably just a consecutive numbering of your auction lots as they are pulled out of the database."
Roy, my 209 image uploads don't have anything to do with auctions or other sales. All except my avatar image upload were images that I posted in various forum discussions. I don't understand why those thumbnails were deleted. How much space could possibly be saved by deleting thumbnails?
The first 29 thumbnails in the list of 209 are still visible. The avatar image at the end of the listing of 209 images is also visible. However, the 178 thumbnails in between are all missing. Each thumbnail box has the word "thumbnail" in it.
Tom
Login to Like this post
"I no longer collect, but will never abandon the hobby"
BuckaCover.com - 80,000 covers priced 60c to $1.50 - Easy browsing 300 categories 02 Nov 2017 06:25:24pm
re: Problems With the Number of Images on the Stamporama Server - Discussion
"I don't understand why those thumbnails were deleted. How much space could possibly be saved by deleting thumbnails?"
There were 375,000 thumbnails totaling 4.8GB that were deleted.
The basic problem was that thumbnails for different purposes were indistinguishable from one another. They were all in the same directory, and simply named by file numbers in the order of upload. Thumbnails were distinguishable because of the extension *_tn.*, but beyond that, there was no clue to their purpose.
It wasn't just the space used by the /upload directory, but the number of files that had to be managed. There were over 1 million files in a single directory. We have now trimmed that down to slightly over 300,000.
Thumbnails for forum posts serve little useful purpose anyway. The only place they show up is in the member's area that list your uploaded images, and the only reason I can see for listing them is to allow deletion, which in my mind, should not be allowed anyway, as deletion of an image effectively negates any point in archiving the original discussion.
As for copying the image to a new discussion, for the very few times that that might be desirable, it's unfortunate, however you can always go back to the original discussion and capture the image filename by right-clicking and then using that filename in an image tag in a new post. Alternatively, upload the image again from your files.
In the end, we had to do a clean-up job. Maybe we lost a few things, but so far nothing really important. We learned some things, and maybe we will do a few things a little differently in future, but keep in mind that nobody is paid, and all suggestions for changes require somebody to spend their time making them happen.
Roy
1 Member likes this post. Login to Like.
"BuckaCover.com - 80,000 covers priced 60c to $1.50- 10,000+ new covers coming Tuesday June 1"
re: Problems With the Number of Images on the Stamporama Server - Discussion
"Thumbnails for forum posts serve little useful purpose anyway."
Speak for yourself, Roy. It's not very likely that we all use SoR in the same manner. Those thumbnails have been extremely helpful in my efforts to share images and associated information, especially in long-running forum threads here at SoR.
The thumbnails are were a management tool for those of us dealing with hundreds of images. If I could click on the file name in my list of uploaded images and view the uploaded image, then I could see why you might think the thumbnails are of little use. But that's not possible, is it?
I've been sharing digital images online in various venues for about 15 years and at SoR for 2½ years. I've always had direct access to either the file itself or a thumbnail associated with the file name. File names alone aren't a very useful tool when I'm trying to keep track of what I've already uploaded and when trying to plan future associated uploads.
I fully understand the file storage problem that SoR is dealing with, but I really hate when unintended consequences disrupt and complicate system users' ability to share information effectively. This was a step in the wrong direction from my perspective.
Tom
Login to Like this post
"I no longer collect, but will never abandon the hobby"
re: Problems With the Number of Images on the Stamporama Server - Discussion
Tom,
You are missing one very important point here. It is Roy who is paying on a monthly basis to hold all those images on his server. This isn't Facebook or Google who are making lots of money from advertising. This is just a club, paid for by just one person, Roy.
re: Problems With the Number of Images on the Stamporama Server - Discussion
Tim, I'm not missing that point. I understand how important Roy's contribution is, not to mention yours and a handful of others who help manage SoR. Roy voiced his opinion regarding the lack of value in the thumbnails and I wanted to point out that his personal view regarding the lack of value in the thumbnails isn't necessarily shared by all members.
My sense is that SoR serves two relatively distinct purposes: The forums and the sales. I think Roy is saying that distinction wasn't taken into account when the file system for the thumbnails was set up. As far as I knew, the clean up was to focus on the image files for the auctions and sales. That's what the discussion focused on as far as I could tell. So I initially assumed my missing thumbnails was a temporary problem that would/could be fixed. I'm disappointed to learn that isn't the case.
I realize you and Roy and others may not understand why the thumbnails are important to the way I work, and I realize there was a need to deal with the vast amount of storage the images were occupying. Several of us here were at StampWants when all of our hard work in posting and sharing philatelic information in the Stampwants forums was thrown out by the site owner. That was hard for all of us to take and the sense of community was forever lost. As a result, I'm more than a little sensitive to seeing changes made that adversely affect my effort to share information here at SoR.
I've been creating thousands of Photoshop images here in my home office since the late '90s. I've had to delete files because I couldn't afford additional storage space. I've lost or misplaced files from time to time due to overloaded or crashing hard drives. When I began, backups went onto CDs. Then DVDs. I've still got a couple hundred of those (2 copies of each) stored here. I have a pretty good understanding of what you're dealing with on a larger scale. I just hope you can find a way to avoid removing my thumbnails in the future since none of them will be related to auctions or sales books.
Tom
2 Members like this post. Login to Like.
"I no longer collect, but will never abandon the hobby"
re: Problems With the Number of Images on the Stamporama Server - Discussion
" It is Roy who is paying on a monthly basis to hold all those images on his server."
I'm also a person who provides services to a hobby and regularly gets clobbered by people who put the service in the same category as if they were at Walmart. I fully understand and respect Roy's benevolence to this board. When I get the notice of a new upload of covers to his business, I try to make a purchase as a thank you. Everyone should.
re: Problems With the Number of Images on the Stamporama Server - Discussion
Tom, this thread has been unavailable and MIA for quite a while and I hadn't been able to view your comments until a few minutes ago.
"...gets clobbered by people who put the service in the same category as if they were at Walmart."
You think I "clobbered" Roy?!?
Roy and/or Tim were tackling an SoR problem and let us know they were working on a change that would remove older thumbnails associated with approvals and auction books. As far as I can tell, they neglected to mention thumbnails associated with the forums. When the bulk of my thumbnails disappeared, I brought this to their attention, thinking it was a temporary glitch that could be fixed. Roy responded by telling me the removal was part of the thumbnail removal plan and my missing thumbnails couldn't be recovered. He went on to tell me my thumbnails "serve little useful purpose anyway." I respectfully disagreed. The loss of my forum post thumbnails is an impediment to my ongoing efforts to share information in the SoR forums. Roy is doing what he needs to do to manage SoR.
I'm disappointed the full extent of the changes wasn't made clear in the beginning and that the changes are adversely affecting my sharing information via the SoR forums, but that's life. Things change. I spent a couple of decades working to effect change in a very bureaucratic organization. Communication matters. Some successes. Some failures. Some members of the organization benefited and some didn't. An unfortunate fact of life.
Tom
1 Member likes this post. Login to Like.
"I no longer collect, but will never abandon the hobby"
A Service Dog gives a person with a disability independence. Never approach, distract or pet a working dog, especially when (s)he is in harness. Never be afraid to ask questions to the handler (parent). 01 Dec 2017 07:53:50pm
re: Problems With the Number of Images on the Stamporama Server - Discussion
Some images missing from the KGV collection thread - or perhaps it's only my computer not showing them.
Login to Like this post
"Let's find a cure for Still's Disease, Breast Cancer and Canine Addison's Disease. We CAN find a cure and save lives!!"
