There's a bug with the Mac OS X Server 10.5.7 Apple Filing Protocol (AFP or "Mac filesharing") process where (perhaps under certain conditions) it fails to restart logging after an arbitrary period of time as specified in /Library/Preferences/com.apple.AppleFileServer.plist (which is typically edited via the Server Admin.app tool).
This bug appears to persist in Mac OS X Server 10.5.8.
For example, if you set AFP to rotate logs every 14 days, the system should compress and roll over logs located in /Library/Logs/AppleFileService every-other week. However that appears not to happen. It will begin to log after the AFP service is restarted, though that's a drastic action.
I presume that /usr/sbin/AppleFileServer handles it owns logs, rather than /usr/sbin/newsyslog, which deftly takes care of old-school logs like ftp.log, system.log and ipfw.log. It's initiated every midnight by the the launchd item /System/Library/LaunchDaemons/com.apple.newsyslog.plist. (It's worth noting that this is not the same as asl, or Apple System Log.)
Anyway, here are three workaround ideas to consider (certainly there are more).