We are excited to announce the release of Alluxio 2.9.2! This is an edge release for Alluxio open source on top of Alluxio 2.9.1, with various bug fixes, documentation, and improvements.
HIGHLIGHTS
Default TTL behavior change
The default TTL behavior previously was to delete the data from both Alluxio and the underlying storage. This is now changed to only deleting the data from Alluxio, leaving the persisted data untouched. See more details in the corresponding github issue and PR.
Prevent recursive metadata sync from getStatus call on a directory
If a metadata sync is triggered by a getStatus call for inode type NONE, the previous behavior was to trigger metadata sync recursively through the directory’s child nodes. The new behavior does not recursively perform metadata sync, only syncing the target of the getStatus call. If the previous behavior is desired, set the property key alluxio.master.metadata.sync.get.directory.status.skip.loading.children to false.
IMPROVEMENTS AND BUGFIXES SINCE 2.9.1
Notable configuration property changes
| Property key | Old 2.9.1 value | New 2.9.2 value | PR | ||||
| alluxio.underfs.eventual.consistency.retry.max.num | 20 | 0 | #16887 | ||||
| alluxio.user.file.create.ttl.action | DELETE | DELETE_ALLUXIO | #16823 | ||||
| alluxio.master.metastore.rocks.checkpoint.compression.level | -1 | 1 | #16801 | ||||
| alluxio.job.request.batch.size | 20 | 1 | #16802 |
Benchmark
- Fix the client stressbench concurrency problem (#16934) (cbff62)
- Add local read benchmarks for PagedBlockStore (#16804) (f9cb6d)
Build
- Fix build error when revision is shorter than 8 chars (#16888) (c4b9ec)
- Add maven build dockerfile with jdk17 (#16885) (aea58c)
- Support Maven reproducible builds (#16881) (6400f4)
- Replace ManagementFactoryHelper with ManagementFactory for java11 (#16874) (782b38)
Client
- Remove the restriction of UFS for local cache (#16819) (02bc22)
- Add CapacityBasedDeterministicHashPolicy (#16237) (c14861)
- Implement unbuffer interface for HdfsFileInputStream (#16017) (903269)
- Change default TTL action from DELETE to DELETE_ALLUXIO (#16823) (7aa8b6)
- Reduce useless async cache request when read from remote worker (#16313) (99923c)
- Add cache TTL enforcement (#16843) (6ca7d1)
- Add the two-choice random cache eviction policy (#16828) (530e72)
Configuration
- Support dynamic update HeartbeatThread tick interval (#16702) (7f6baa)
- Fix updated config not notifying config registry (#16764) (407cf1)
Docker and K8s
- Respect MaxRamPercentage in container (#16940) (492cc7)
- Add buildx support for build-docker.sh (#16822) (98dba4)
Job Service
- Fix cli distributed command hang issue (#16786) (59f660)
- Fix logging to capture jobControlId (#16780) (412e54)
Log
- Fix debug log in FileSystemContext#reinit() (#16826) (fb07ea)
- Fix logging to capture jobControlId (#16780) (412e54)
Master
- Improve LS by allowing to omit UFS and mount info (#16893) (3f8c79)
- Fix metadata sync behavior when descendant type is NONE (#16935) (8ef0a4)
- Make standby master keep MountTable up-to-date (#16908) (a5d57e)
- Make workers register to all masters (#16849) (ebeac4)
- Add allowOnStandbyMasters option for version grpc endpoint (#16890) (da6abf)
- Fix file fingerprint to use atomic get content hash of uploaded file (#16597) (00da77)
- Make version service return unavailable on standby masters (#16854) (7399c3)
- Support rm directory and reset DirectChildrenLoadedState (#15647) (778f9e)
- Support gRPC on standby masters (#16839) (325f36)
- Add disable are descendants loaded property (#16813) (5a7cfe)
- Use async journals for block id journal entries in metadata sync (#16529) (9f91f9)
Metrics
- Add metrics sink to job master (#16899) (73f3ce)
- Fix Worker.ActiveClients is negative when load from ufs (#16784) (60d515)
- Add metrics for in memory data structures (#16818) (ee6c23)
- Fix metric worker.cacheBlockSize increases when cache already existed (#16791) (945b10)
S3 API
- Fix incorrect flag passing into delete op (#16941) (a02c5f)
- Add rate limit for s3 proxy (#16866) (f0e9a5)
- Support overwrite option in createFile (#16886) (16ff65)
- Introduce a rearchitectured S3 proxy service (#16654) (f3eca2)
CLI
- Fix head and tail commands read less than expected (#16928) (c4a700)
- Fix free worker command bugs (#16458) (ff0a6d)
- Fix cli distributed command hang issue (#16786) (59f660)
UFS
- Update retry policy for object stores (#16887) (47320b)
- Merge delete requests to Optimize move directory perf on objectstore (#16527) (7ab257)
- Refactor s3 low level output stream and support OSS and OBS (#16122) (0b4796)
Web UI
- Display more information in WebUI Masters (#16636) (8502fb)
- Support display master system status from master ui (#16779) (fb9e88)
Worker
- Support remove blocks on worker for pagestore when free/delete file (#16895) (669f80)
- Fix free worker command bugs (#16458) (ff0a6d)
- Add local read benchmarks for PagedBlockStore (#16804) (f9cb6d)
- Merge local cache invalidation (#16841) (506a0f)
- Split BlockStoreEventListener.onCommitBlock() (#16777) (6fc070)
ACKNOWLEDGEMENTS
We want to thank the community for their valuable contributions to the Alluxio 2.9.2 release. Especially, we would like to thank:
Deepak Shivamurthy (deepak-shivamurthy), Haoning Sun (Haoning-Sun), Ce Zhang (JySongWithZhangCe), Kaijie Chen (kaijchen), Nandeeshvar Porko Pandiyan (nand-porko), Vimal (vimalKeshu), Xinran Dong (007DXR), Yaolong Liu (codings-dan), bingzheng, (bzheng888), fanyang (fffanyang), humengyu (humengyu2012), linda (wenfang6), Baolong Mao (maobaolong), Lei Qian (qian0817),Xinyu Deng (voddle), Yangchen Ye (YangchenYe323), yiichan (YichuanSun), Tianbao Ding (flaming-archer)
Enjoy the new release and look forward to hearing your feedback on our community slack channel.