deduplication

Similar Questions

• Deduplicator bookmark does not work with Firefox 35. What should I do?

  It worked before I installed the latest version of Firefox. Now whenever I press start remove duplicates, nothing happens. Please help me understand what to do. I use Mac 10.9.5. Thank you!

  https://addons.Mozilla.org/en-us/Firefox/addon/bookmark-Deduplicator/

  The last review of the user of this extension mentions that he does not work in Firefox 35. Other than to wait that the developer of this extension realizes its extension is broken and fix it on what you can do, is looking for a replacement / extension of replacement with this feature. Unfortunately, the developer did not provide a contact email link to encounters with bugs or comments and did not provide a "home page" with the details either.

• Someone understands technology "Deduplication"?

  Hi people,

  I ask this question here because I agree with a company of Cloud backup and am unable to get a satisfactory response from their support or their product Forum - in fact they won't even after my question on the forum (must be approved by the moderator).

  I have a problem with the Client-Cloud/Backup software, it keeps running "integrity checks", 'fixing' of data errors and re - running and to find more/same mistakes ad infinitum.

  If I re - install the client software and the nature of the companys cloud system I can't re-synchronization with the existing backup. -J' I download all of the backup from scratch once again.
  When I studied this problem on their forum, I found mention of 'the' file de-duplication technology. -The moderator suggested/said a poster who was complaining about the same question (re - download backups because you can not re - connect to an existing backup) of duplicaton technology would make a re-upload later seem a lot more quickly.

  Hmmmm think I, which doesn't seem wise, - surely if their server uses deduplication technology (that I understand to be a way to eliminate duplication of files on their storage system to save space) and THEN... He must have a complete copy of the file to compare with? or in other words, we must always load your entire backup?

  The substance of my question is the following:-"deduplication" takes place on your computer (i.e. in the client software) or on the server where the storage / hard drive space is located. ?

  The Cloud company cannot or will not answer this...
  does anyone here know?

  Hi Nicolas,.

  The question you have posted is better suited for Windows 8 IT Pro category in the TechNet Forum. Post the question on the link provided below for assistance.

  Windows 8 IT Pro

  http://social.technet.Microsoft.com/forums/en-us/category/w8itpro

• How to make a rule of deduplication?

  Hello world

  It is probably a very simple question for a large number of Topliners here, but I'll try to find how to run a rule of deduplication.  I searched and I can't find any document which will specify the exact step on in fact runs the rule.

  In the overview of deduplication post (http://topliners.eloqua.com/community/know_it/blog/2012/07/26/overview-of-deduplication-rules) rules, there are many examples, and he talks to review the resulting data (source matches, source non-matches, matches of destination, dest. non-matches).  But the post covers how and where in Eloqua you execute this rule.

  What I'm missing here?

  For simplicity, I want to know if there are several people with the last name "Carroll".  I want to be able to execute this rule of deduplication and eliminate anything but a case of it (again, for simplicity, say I want one that has the city = "Denver").  I built this rule (correctly, I think).  I use the program generator to run a rule of deduplication?  I did, but I don't know what to do after the action step run Dedupe rule.  See the screenshots for details

  Thank you

  Just as a follow-up, I have some more information on these rules.

  First, they are executed in the programs, in which you perform actions against recordsets resulting (such as the removal of the update of a field program, etc.).

  And secondly, deduplication rules little replaced by the provisions of Match.  In other words, game rules are a more recent version of the features you want, when you think about deduplication rules, well that dedupes let perform actions against the Recordset without correspondence.

• vSAN deduplication (Standard or advanced)

  Is anyone know when VMware finally a vSAN version which has the peculiarity of deduplication if it will be included with the Advanced Standard and/or vSAN license?

  Thank you!

  Deduplication and compression doesn't work with the all-flash, so license advanced is a must.

• By VM or datastore deduplication deduplication?

  I had a discussion with a colleague and we have diverged on how we thought deduplication worked for VDR and VADP.

  The method of deduplication is applied individually on each virtual machine data blocks or is it applied on all dedupe datastore?

  If we treat blocks of each virtual computer as a single entity, unless a single virtual machine has identical blocks it be no de-duplication.  However, if the de-duplication is applied across all datastore deduplication, so even if a single virtual machine does not duplicate blocks, there may be still space-saving if an another VM has blocks that are identical to the blocks 1 VM stored on the same datastore dedupe.

  I checked the documentation for VDR and found the following:

  "Because the support of VMware store the results of multiple backup jobs to use deduplication even store, . maximize deduplication rates, ensure that the similar virtual machines are backed up to the same destination.  While the backup of virtual machines in the same store of deduplication can produce an increase in spacesavings, similar virtual machines do not need to be backed up by using the same work. Deduplication is evaluated for stored virtual machines, even if some are not currently saved. »

  It seems to me that it's suggesting that we backup similar to the same datastore dedupe VMs to maximize our spacesavings.  For me, it also means that deduplication is applied to all dedupe datastore, instead of treating the VMs is isolated objects.

  I don't know if there's some blog out there that contains more detailed information, but the official documentation does not specifically mention one way or the other.

  Can someone tell me please in the right direction?   My colleague and I received different answers from what we thought to be reliable sources, but obviously both of our sources may not be correct.  References of official document would be fantastic, but before any reference would be greatly appreciated!

  Thanks in advance for your help.

  Robert

  I don't know a documentation which specifically mentioned the "global deduplication. However the next part of the Guide of Directors should clarify / prove it.

  The operating system files are often identical between the virtual machines running the same operating system. To maximize deduplication,
  Save a similar virtual machines to the same destination.

  André

• LabManager and NetApp deduplication ratio

  We will go to the storage of one EMC to a NetApp device soon. Does anyone have figures as to how much deduplication allows, in a link to environment related clone Lab Manager? This is of course design. Does anyone have a recommendation of NetApp model for about 20 TB of usable space on disk (for LM) SAS? Overall, we expect twice the size.

  In all of our six warehouses of Lab Manager totaling about 10 TB of usable space, we see a range of 35 to 60 percent of the savings from de-duplication.  Deduplication rates are really based on how similar VM and activities you do on the same volume of NetApp (data store) which represents for various numbers see us.  On savings high volumes, we tend to have a number of very similar configurations of Lab Manager library, and from them much deployment used by our body of development, more facility similar if not identical copies of the same software in each copy.  Lots of duplicate blocks = more savings in case of deletion of duplicates.

  Data warehouses are hosted on a mixture of old fibre channel and SATA shelves and a new tray SAS.  Most of the storage is served on a three year old model FAS3040 server, only the latest SAS storage while used a FAS3160.  In our environment, two reporters are in fact a mixture of NAS services generalist as well as the storage of virtualization.  This is not ideal and caused a lot of problems (especially the storage virtualizaiton overwealming all other use...).  In general, the model FAS3160 is very well - much better than the 3040.

  One thing to consider is that the scans of deduplication, during execution, can consume a good amount of the file server CPU resources.  It is one of the biggest problems we have on the 3040 when it tries to radiation upward the two largest 2 TB FC data warehouses.   Offers of deduplication of value is exceptional, but you definitely need to ensure sufficient system capacity to deliver the savings achieved in parallel with the normal I/O to the infrastructure service attached to the spinning.

• Deduplication before vSphere VMware data recovery 4

  Hi people,

  I decided to give a try to the Vmware Data Recovery product. I have recently updated our 3.5 ESX to vSphere 4. I installed the plugin for recovering data and the device, and I think that it does not work as it is supposed to, but I have a question.

  I read the documentation about the product to recover data and on page 8 of the admin guide, there is a note on the optimizations not applied to virtual machines created with vmware vSphere products 4.

  I vmotioned my machines windows and upgarded their vmware tools and equipment of 7 virtual machine. I was wondering these machines were created with ESX 3.5 and now that I've migrated to vSphere 4, doesn't that mean that deduplication will not take place with data recovery products when I'm in my data store backup? or am I misunderstanding this?

  Thank you.

  Hello and welcome to the forums.

  Once you upgrade older VMS v4 to v7 hardware virtual machine, then they are the same.  Discover the answer of amina in this discussion.

  Good luck!

• FC LUNS and data deduplication (NetApp)

  I'm curious to know how we could take advantage of NetApp deduplication of data, if we use FC LUNs.  I realize with NFS, that the issue is much simpler.  But suppose that we do not have NFS for the moment...

  If we present a 500 GB LUNS ESX and make than in a VMFS volume, the operating system has the volume.  Then, we put 400 GB of data on this volume.  The back-end that should say 300 GB of data deduplication.  But wouldn't VMware see still 400 GB to a volume of 500GB used?  Where is space saving occurs?

  The LUN is still 500GB in the NetApp, so we do not see the savings there either (we cannot distribute more LUNS with deduplication can us?)

  Basically how to make use of the data deduplication space savings, if you use the FC LUNs.

  I spoke to my engineer to verify that we were researching the product and we currently use Iscsi, so it's of great interest to us (although NFS is also considered to be heavy)

  Response to NetApp is fairly simple

  (1) de-duplication works on tier (FC and iscsi) block to block level deduplication content or netapp 'Knowing' the content is irellivent because it is just block on the disk and all reguardless of 2 blocks of content that are identical to de-dupped (not supporising as other backup technologies do the same thing with encrypted files)

  (2) to regain space dedup you must uncheck the "placeholder" otherwise reserves lun the reguardless of it space is consumed (and it will not be because of dedup), you then return to space by multiple LUNS on the flexvol

  (3) the dedup arrives on the level flexvol reguardless of is it the LUN iscsi, NFS, CF, or any mixture of the 3

  (4) the hosts will never see the economies of block-level storage to reclaim the space must be made through #3

  I asked to send the white paper of my way, but according to their engineers is no obstacle.

  Those that I found there

  http://blog.scottlowe.org/2008/04/24/using-NetApp-deduplication-with-block-storage/

  http://blog.scottlowe.org/2008/05/20/provisioning-LUNs-for-use-with-deduplication/

  http://communities.NetApp.com/docs/doc-1192;JSESSIONID=7177122E64AA32F30DD92CEE995AC70E

  Well, it seems to work assuming that you make the correct configuration... documentation shows this way.

  Anyway, we are always looking for NFS twards as the host is able to SEE that reclimation of space (that and that we have an ISCSI network using software initiators it won't cost us more to do)

  Jered Rassier

  ##If you found my post answered your question or helpful please mark it as such.

• How to check if the deduplication works correctly

  Hi all
  I understand that deduplication securefile is for the same column. Suppose if I have 2 identical data in a column, say one, with the size of 4000 bytes, the deduplication makes stores only 4000 bytes instead of 8 000 bytes table. From sqlplus, how can I check if this particular column takes only space to 4000 bytes instead of 8000?
  
  Kind regards
  Val

  Valerie Debonair wrote:

  How can I check if this particular column takes only space to 4000 bytes instead of 8000?

  You can't exactly check without looking at the discharge of the LOB segment. It is easier to just check LOG segment does not increase when you insert the same LOB values:

  SQL> DROP TABLE T1
    2  /
  
  Table dropped.
  
  SQL> CREATE TABLE t1(
    2                  C CLOB
    3                 )
    4    LOB(C) STORE AS SECUREFILE
    5  /
  
  Table created.
  
  SQL> SELECT  SEGMENT_NAME
    2    FROM  USER_LOBS
    3    WHERE TABLE_NAME = 'T1'
    4      AND COLUMN_NAME = 'C'
    5  /
  
  SEGMENT_NAME
  ------------------------------
  SYS_LOB0000077372C00001$$
  
  SQL> INSERT
    2    INTO T1
    3    SELECT  LPAD('A',10000,'A')
    4      FROM  DUAL
    5  /
  
  1 row created.
  
  SQL> COMMIT
    2  /
  
  Commit complete.
  
  SQL> SELECT  BYTES
    2    FROM  USER_SEGMENTS
    3    WHERE SEGMENT_NAME = 'SYS_LOB0000077372C00001$$'
    4  /
  
       BYTES
  ----------
     1245184
  
  SQL> INSERT
    2    INTO T1
    3    SELECT  LPAD('A',10000,'A')
    4      FROM  DUAL
    5      CONNECT BY LEVEL <= 100
    6  /
  
  100 rows created.
  
  SQL> COMMIT
    2  /
  
  Commit complete.
  
  SQL> SELECT  BYTES
    2    FROM  USER_SEGMENTS
    3    WHERE SEGMENT_NAME = 'SYS_LOB0000077372C00001$$'
    4  /
  
       BYTES
  ----------
     2293760
  
  SQL> DROP TABLE T1
    2  /
  
  Table dropped.
  
  SQL> CREATE TABLE t1(
    2                  C CLOB
    3                 )
    4    LOB(C) STORE AS SECUREFILE(
    5                               DEDUPLICATE
    6                              )
    7  /
  
  Table created.
  
  SQL> SELECT  SEGMENT_NAME
    2    FROM  USER_LOBS
    3    WHERE TABLE_NAME = 'T1'
    4      AND COLUMN_NAME = 'C'
    5  /
  
  SEGMENT_NAME
  ------------------------------
  SYS_LOB0000077375C00001$$
  
  SQL> INSERT
    2    INTO T1
    3    SELECT  LPAD('A',10000,'A')
    4      FROM  DUAL
    5  /
  
  1 row created.
  
  SQL> COMMIT
    2  /
  
  Commit complete.
  
  SQL> SELECT  BYTES
    2    FROM  USER_SEGMENTS
    3    WHERE SEGMENT_NAME = 'SYS_LOB0000077375C00001$$'
    4  /
  
       BYTES
  ----------
     1179648
  
  SQL> INSERT
    2    INTO T1
    3    SELECT  LPAD('A',10000,'A')
    4      FROM  DUAL
    5      CONNECT BY LEVEL <= 100
    6  /
  
  100 rows created.
  
  SQL> COMMIT
    2  /
  
  Commit complete.
  
  SQL> SELECT  BYTES
    2    FROM  USER_SEGMENTS
    3    WHERE SEGMENT_NAME = 'SYS_LOB0000077375C00001$$'
    4  /
  
       BYTES
  ----------
     1179648
  
  SQL> 
  

  SY.

• Just imaginary deduplication or only after replication or archiving?

  Anyone else running appassure noted that the meter of deduplication is apparently imaginary? I'm going to prosecute and wait 3-4 weeks to get anywhere, I know, but I just did some calculations on all my carrots and it is quite obvious that the total of protected data * compression = used space on the repository. So what is deduplication meter 47% indicating?

  Although on my replicated core is the used space is much closer to what we would expect with compression and deduplication... well on this basis, the percentages of compression and deduplication are completely different and don't match anything I can find.

  That's it, after that I have been instructed by the support to rebuild the repository because our total protected data were lower than the space used on the repository if our deduplication and compression read more than 30% each.

  Hello

  Two different counters of compression and deduplication

  1. deduplication - indicating how many records have been structure since the last job "to maintain the repository."

  2. compression - indicating the difference between protected and space used in the repository, including deduplication.

  NOTE: With each volume image transferred to the volume, these metadata metadata is no compression and not deduplication.

  Thank you

  Anton Kolomiiets.

• No deduplication on several images in database?

  We have been using AA for a while now (approximately 18 months) and there is one thing that has really bugged me, but do not know if it is something that is unique to us.

  Does anyone else have the issue where it seams are not dedplication at all when AA is one of it is re-baseimages arcana?

  From our experience, if you have, say a 2 TB file server, and AA decides to take another base image, then it just goes straight up use an extra 2 TB of the Repo.  It seams that this would be the perfect location for when deduplication should eliminate most of these redundant data almost as he takes a second database same (especially).

  Curious to see if anyone else has noticed the same thing?  It's a trap for new players, as it blows completely to your needs of size of potential repo depending on what type of retention that you want to keep the disk, and you cannot accurately measure when AA is going to decide to do a base image, it is very easy to get caught short so to speak.

  Not alone.  You must be on 5.4 + and set the size of your cache of deduplication

  certainly a hunt witches for newcomers.  Note, requires RAM and you've probably already lost a lot of your deduplication on a base of 2 TB image.

  en.Community.Dell.com/.../20665224

  en.Community.Dell.com/.../20613532

  en.Community.Dell.com/.../20707155

• Best way to erase 'Music' of the iPhone and Resync with my newly deduplicated iTunes (I removed 30 GB of duplicates with "Tune Sweeper")

  Hi, I recently decided to clean up my years of iTunes duplicates.  "Tune Sweeper" ($24.99) reduced my 180GB to 150 GB.

  I don't want my iPhone (50 GB free) existing MUSIC to restore my iTunes on my PC.

  Can I just remove all the current MUSIC from iPhone and resynchronization,

  I go into the settings (storage) and deleted xxGB music on my iPhone but all my songs are still playable.

  I'm willing to pay for advice, if this forum is not able to help.

  Any suggestions?

  Connect the phone to iTunes on the computer that you sync with and click the music option. Uncheck the option "Sync music" and sync. This will delete all music from your device.

  Then go back to the music option and select what you don't want sync'd music on your device.

  See you soon,.

  GB

• Deduplication of files Total possible

  Under libraries, there is a folder of Documents.  In this case, there is 7,150 elements.  Those that appear to be duplicates of files in multiple subfolders I have under My Documents.  Are these duplicates?  When I launch Windows Backup, I support all these files twice?  Can I delete the entries under the Documents folder without affecting the files in the folders under My Documents?  Thank you!

  So, what is the answer to my original question.  Delete the 7 150 showing under Documents?  They are still saved uber My Documents files?

  No, if you remove them, they will be removed from your computer.  They only exist in one place and it's

  C:\Users\\My Documents.  When you delete them from the library, they will be removed from

  in the world.  Like I said they are not duplicated in the library.  The library has just points to the current storage location.

  They not be saved twice.

• Deduplication of unwanted files

  Recently, I imported several folders (and subfolders) in LR.  I organize my photos by year and month (and then sometimes during the month).

  So, for the 2015 folder, I currently have a subfolder, tagged 1.15

  On my hard drive (IE in windows Explorer), there are no files in the folder labeled 2015 unless they are also 1.15; so as you can see in the screenshot below, everything that appears in 2015 is 01.15 folder without and separate photo files

  

  But once imported into LR, all files of 2.15 photos appear twice: once in 2.15 and in 2015; as you can see in the snapshot, the photos are present in the folder of 2015.

  

  This is a problem because the subfolder help me to organize and separate files; more importantly when I create several subfolders in the folder of the month.

  How can I avoid this and the format of the mirror window File Explorer?  I know these aren't really a copy of the files and they are simply indexed in LR, but this isn't always ideal.

  Lightroom is showing the total number of photos in the parent folder and all its subfolders.

  If you do not want to see the countdown for parent folders, and then right-click on the sign more in the right side of the files Panel and uncheck the option "show pictures in subfolders.

  

  So, if you don't have all the photos in the folder "2015", it will just show a zero and show count of 144 for the folder "1.15 '.

  For example: 'show pictures in subfolders' is activated/checked:

  

  With unmarked:

  

  However when reduce you the 2015 folder in the folder view, it will just show a count of zero

  

  Personally, I find the opportunity to have photos of the less complicated number of records show

  There is no pictures in the folder of the Nature, but its subfolders have 573 photos. Also note the change to the County for the record of birds (only 37 in this folder, and the rest are in the sub folders)

  Bruce