SDL Web - How to cleanup files left over by publishing

State files are not being removed from incoming folder even though the cleanup param value is set to true.  
This eventually causes content publishing to fail with error such as the below

Transport failed: Could not transport TCMID.Content.zip using HTTPS

 

By default, files that are generated during the publishing process are automatically deleted / removed. In some cases, this might not happen due to various reasons such as services/applications that are down unexpectedly and the files might not get removed as it should.

The below steps should be performed to ensure the publishing and/or deployer processes run efficiently as leftover files could impede the process.
Steps:

1. First, ensure there is no outstanding publishing job in the publishing queue. Outstanding publishing jobs are jobs that are not Success or Failed. In other word, jobs that have status such as Waiting for publishing, Rendering, Deploying, Committing deployment.
   If those outstanding jobs remained stuck, go ahead and remove them from queue. (will need to re-submit the item for publishing again once the cleanup is completed)
2. On the Content Manager server (or publisher server), look into the folder called transactions under the ..Tridion\bin folder and remove all files under that folder:
   
   
   
   NOTE: The transactions folder is defined in the cd_transport_conf.xml. If you have modified this folder to be a different folder, then go to that folder and remove files.
   
   
    
3. On the deployer application, look in the incoming folder and remove all files seen there (except the file called meta.xml and folders, if any)
   NOTE: The incoming folder is defined in the cd_deployer_conf.xml
   
   
    
4. On the Content Manager server (or publisher server), restart the Transport and Publisher service
5. On the deployer server, restart the deployer application (either by restarting IIS, or by restarting the deployer application pool)
   See KB article For Tridion Sites 8.5, what happens to publish transactions that are in progress when the Deployer is restarted?
6. That’s it. The cleanup process is completed. Re-submit items for publishing if they have been removed from publishing queue earlier.

To remove publishing trace files, some customers will run a script periodically to cleanup old files.  There is no productized script for this, but an administrator could for example run a script for example which removes all files from the Deployer incoming folders, which leaves files that are older than 7 days, and runs during non-business hours.  Note troubleshooting or implementation of such a script is outside the scope of SDL Support.
It's advisable to verify that any AV software or similar monitoring applications do not scan the Deployer incoming folders or the CM transactions folder.  It's possible that an AV scan could overlap with a publishing transaction and prevent a trace file from being removed as expected.  See KB article For Tridion Sites 9.5, what folders should be excluded from anti-virus scanning to avoid negatively impacting performance and functionality while scans are taking place?

Also see KB article Deployer incoming folder not being cleaned up