Rsync requirements

Greetings !!!

This mail is regarding a specific requirement about rsync that we are trying to achieve. We have tried to achieve this by using various rsync options. However, we are encountering different problems with different rsync options.

Background:
• We have a process (running on AIX) logs from which are getting logged in A.log (present in logs directory).
• A.log gets rotated to A.CURRENT_DATE_TIME.log once it reaches 100 MB and new A.log is created.
• We are transferring these logs to a centralized server using rsync. We are using rsync on the complete logs directory.
• INODE of the files on the source server and the destination server are different.
• Once the logs are in the centralized server, the idea is to get these logs read/indexed by a centralized log process which will pick the input from this centralized server.

Problem:
• Although, A.log (destination server) is given as input to the centralized log process, it considers the INODE of the file and not the actual filename.
• So, when the A.log file gets rolled over, the new A.log has a new INODE which is not detected by the centralized process. This was happening when we were using -u –r –t options with rsync. So in this case, INODE of the file was changing with everytime rsync happened and also when the rollover happen. Hence, the process stops indexing as it looks for the old INODE which is not present.

• The idea is to use rsync with a combination of options that would not change the INODE of the file at the time of rsync but should change the INODE at the time of rollover when A.log rotates to A.CURRENT_DATE_TIME.log. So, in order to achieve this, we included the –inplace option and we are able to retain the INODE at the rsync and INODE changes at the time of file rotation. However, it gives us a different issue now where file name doesn’t change and always remains A.log. So once the process is done indexing the A.log, it halts.

It would be great if someone can suggest something that could help us in achieving the mentioned requirements.

Regards,
Puneet Sinha
Middleware Administrator


Source: unix

Leave a Reply