Twitter ã§ã® 2幎
å㯠Twitter瀟㮠Build/Bazel Migration ããŒã ã§ã¹ã¿ããã»ãšã³ãžãã¢ãšããŠå€åããŠãããä¿¡ããããªããã㪠2幎ã®åŸã2022幎11æ17æ¥ããã£ãŠéè·ãã (äŒæ¥è²·ååŸã®ã¬ã€ãªãã§ãä»»æã§ããããŸãé¢ä¿ç¡ãããåã¯ä»»æéè·åžæã®ãªãã¡ãŒãåã£ã)ãTwitter瀟ã¯ãåç£ç¢ç£šã倿§æ§ããã㊠Flock ãæ§æããå šãŠã®äººã«å¯ŸããŠæº¢ãåºãåªãããšããããªãç¹å¥ãªæåãæã£ãè·å Žã ã£ãããããéè¿ã§çµéšããŠããã®äžå¡ãšãªãæ©äŒãåŸãããšã«æè¬ããŠããã(Flock ã¯ãé³¥ã®çŸ€ããã®æã§ã瀟å ã§ã® Twitter瀟ã®éç§°)
以äžã¯éå»2幎ã®ç°¡åãªæ¯ãè¿ãã ãå°æ¬çš¿ã§ã®æ å ±ã¯ãæ¢ã«å ¬éãããŠããããŒã¯ãããŒã¿ã«åºã¥ããŠãããè²·ååŸããã¡ã®ããŒã ã ãã§ã 10å以äžã®ã¡ã³ããŒã Twitter瀟ãæããã®ã§ãåšç±ã»å å«ã LinkedIn ãããã¡ã€ã«ãžã®ãªã³ã¯ãæ¬çš¿åæã«è²Œã£ãã
EE Build ããŒã
ãŸããBuild ããŒã ã®ä»»åã解説ããªããŠã¯ãããªããšæããå
¬éãããŠããããŒã¿ãåŒçšãããšã2020幎é ã§ãåŒç€Ÿã«ã¯çŽ 2000人ã®ãšã³ãžãã¢ãåšç±ããŠãã¢ããªãã ãã§ãææžãã®ã³ãŒããçŽ2åäžè¡ (çæã³ãŒããå«ãããšãã® 10å) ããããã®å€ãã Scala ã ããPythonãJava ãã®ä»ã®èšèªã®å€§èŠæš¡ãªã³ãŒãããã£ããã³ãŒããã®ãã®ã®èŠæš¡ã¯äžæŠçœ®ããŠãããŠããããŒã ã®æ°ãå€ãã®ã§ãæ¥ã
å€ãã£ãŠããã³ãŒãã®å€åéãé«éãªãã®ãšãªã£ããã³ãŒãããŒã¹ã®èŠæš¡ãšã㊠Twitterç€Ÿã¯æ¯é¡ãç¡ãããã§ã¯ç¡ãããããããã®èŠæš¡ãšãªããšä»ã®ãšã³ãžãã¢ãæ®éã«ã³ãŒããæžãããšãå¯èœãšããããšãç®çãšãããšã³ãžã㢠+ ãããŒãžã£ããæ§æãããéšçœ²ãå¿
èŠãšãªããç¹åãã JVMãã«ã¹ã¿ã åãã git
ããã«ãã»ããŒã«ãCI ãªã©ã管çããããã®éšçœ²ã¯ãEngineering Effectiveness (ããšã³ãžãã¢ãªã³ã°å¹çå課ã) ãšåŒã°ããã
EE Build ããŒã ã¯ã瀟å ã§ Source ãšåŒã°ããŠããã¢ããªããã補åããšããŠæã£ãŠããã2020幎ãŸã§ã¯ããã®ããŒã 㯠Pants ãšããç¬èªã®ãã«ãã»ããŒã«ãéçºããŠããŠããã㯠Google瀟㮠Blaze ãšããå éšãã«ãã·ã¹ãã ã®åœ±é¿åããTwitter瀟ã®éçºç°å¢ãšéçºé床ã«åãããŠåæ 2010幎代ããã® Scala èšèªãµããŒããªã©ã®è±å¯ãªæ©èœã远å ãããã«ãã»ããŒã«ã ã
2015幎ã«ãªã£ãŠ Google瀟㯠Blaze ã®ãªãŒãã³ãœãŒã¹çã§ãã Bazel (èªã¿ã¯ãããŒãŒã«ã) ãçºè¡šããè¿å¹Žã¯å€ãã®äŒç€Ÿããã©ã°ã€ã³ãåšèŸºããŒã«ãæäŸããæŽ»çºãªãã«ãã»ããŒã«ã«è²ã£ãŠããŠããã2020幎ã®4æã« Build ããŒã ã¯ãPants ãã Bazel ãžç§»è¡ããããšãçºè¡šããã
å€§èŠæš¡ã³ãŒãããŒã¹ã«é¢ãã£ãããšãç¡ããã°ãã«ãã»ããŒã«ãæ¡çšããã®ã«ãšã³ãžãã¢ããæ§æãããããŒã ãå¿ èŠãšãããçç±ã¯èªæã§ã¯ç¡ããããããªãã誀解ãæããã«ç°¡ç¥åãããšã2020å¹Žåœæã® Bazel ã¯ãã«ãã»ããŒã«ãšããããã¯ããã«ãã»ããŒã«ãäœãããã®ããŒã«ããããã«è¿ãã£ããçç±ã¯è²ã ãããã1ã€ã¯ Google瀟å ã§ã¯ãããã€åšããªã©ã§å¥ã®ããŒã«ãæ¢ã«ãã£ããšããäºãšãPants ã®è±å¯ãªæ©èœãšå ±ã«é²åããŠãã£ã çŽ2åäžè¡ã®ã³ãŒãããããšããäºãããããã®ãããªæãè¡ãã§ãBuild ããŒã ã¯ãBazel ãžç§»è¡ããããšã§æåŸ ãããé«éåã倱ãããšãªãè±å¯ãªæ©èœãåå®è£ ããTwitter ãå®è£ ãããµãŒãã¹çŸ€ãããŒã¿ãžã§ããå®éã«ç§»è¡ããããšãç®çãšãã Bazel Migration ããŒã ãšãªã£ãã
å
šç€Ÿãäžæ°ã«æ··ä¹±ããããªã¹ã¯ã®ãããããããããã°ãã³ã»ãã€ã°ã¬ãŒã·ã§ã³ããé¿ããŠããã¯ãã¬ãã«ã§ Pants ãšãã¥ã¬ãŒã·ã§ã³ã»ã¬ã€ã€ãäœã£ãŠãBUILD
ãã¡ã€ã«ã Pants ãš Bazel ã®äž¡æ¹ããèªã¿èŸŒãããšãã§ãããšãããŠããŒã¯ãªæ¹æ³ãæ¡çšãããããã¯ãå®è¡æé床ãç ç²ãšããããšç¡ã段éçæ¡çšãå¯èœãšããã
2020
åã Twitter瀟 Build/Bazel Migration ããŒã ã«å ¥ç€Ÿããã®ã¯ 2020幎ã®8æã§ãã¯ã¯ãã³ãåºãåã®æ°åã³ãããŠã€ã«ã¹ã»ãã³ãããã¯ã®çæäžã ã£ããäžçã¯åšå® å€åã®çŸå®ããããé å¿ããããŠããããå㯠2011幎ããåšå® å€åããŠããŠããã®ã§æ £ããããã1é±éç®ã¯ Flight School ã§ã瀟å ã®ã€ã³ã¹ãã©ã¯ã¿ãŒãäžçŽãšã³ãžãã¢ã«ãã£ãŠæè¡ã¹ã¿ãã¯ã瀟å ã®ã«ã«ãã£ãŒãªã©ã®ã¬ã¯ãã£ãŒãäžé±éè¡ããã瀟å¡ç ä¿®ã ã£ãã
Build ããŒã ã¯ãç°åãããã€ã€åžžæçŽ12åãç±ã«æã¡ãããšã¯æŽŸé£ã³ã³ãµã«æ°åãé¢é£ããŒã ãããæ°ååãããšããããšããã£ãŠãããåã¯æ¯èŒçå°ãããªããŒã ã§ããåããããšããªãã£ãã®ã§ãæåã¯å€ããã§æ £ããªãã£ããã ãããæåã®æ°é±é㯠Yi ChengãããããŒã ã«ãªã³ããŒãããã®ãå©ããŠãããããšãèŠããŠãããYi ã¯ããŒã ã®æ±çååšã§ãPantsç³»ã®è³ªåãªãå šãŠã®çããç¥ã£ãŠããŠãçã£å ã«äººãå©ããã¿ã€ãã§ãäžäžå·Šå³ã®äž¡æ¹åã®æ§ã ãªããŒã ãšã®ã€ã³ã¿ãŒãã§ã€ã¹ãè¡ã£ãŠããã
åœæã§ãåã¯çŽ10幎㮠ScalaæŽãæã£ãŠããã®ã§ãæ°å ¥ããšããŠã¯çããæãã ã£ããæåã®æ°é±éã§ãPants çšã®å éšãªã¢ãŒãã»ãã£ãã·ã¥ãµãŒãã¹ã§ãã buildcache ã« Bazel ã®ãµããŒããæ¹ä¿®ããã (buildcache ã®è©³çŽ°ã¯ 2015幎㮠Scala at Twitter åç §)ãèšã£ãŠããŸããš Bazel ãšããŠã¯æé©åããããœãªã¥ãŒã·ã§ã³ã§ã¯ç¡ãããæªããªãåºå ã ã£ãã
次ã«ãåœæãã³ãã³ã«ã㊠Build ããŒã ã® Tech Lead ã ã£ã Ity Kaulãããšè©±ãããããŒã å ã§æã瀟ã§ã®çµæŽãé·ããã©ã³ã¯ãæäžã ã£ã圌女ã¯ã¯ãŒã¯ã¹ããªãŒã ã®æŽçãšãããã®é²æã®ç®¡çã«å¿ããã£ãã1-on-1 (ãã¯ã³ãªã³ã¯ã³ã25åéã®å人ããŒãã£ã³ã°ãäžåžéšäžã«éããããšã³ãžãã¢å士ãäžåžã®ãã®äžåžãªã©ãš2é±éã«1åãååæã«1åãªã©æ§ã ãªã¿ã€ãã³ã°ã§å人ããŒãã£ã³ã°ãè¡ãããã) ã§ãåãã§ããããšã§æŠç¥çã«æãé¢çœãåé¡ã¯äœããšèããŠã¿ãæããã«ãããŒã¹åé¡ (ãå€å å®å®åé¡ã) ã«ã€ããŠæããŠãããã
ãªã®ã§ 9æã«ã¯ãã«ãããŒã¹ãæ°ãç¶ããããã«ãããŒã¹ãšã¯ãäŸãã° { A: 1.0, B: 2.0 }
ãšãã£ãã©ã€ãã©ãªäŸåæ§ã®ããŒãžã§ã³çªå·ã®éåã®ããšã ãPants ã¯ãã³ãã³ãå®è¡æ¯ã« Coursier ãåŒã³åºããããã¿ãŒã²ããããšã«ç°ãªãã©ã€ãã©ãªã®ããŒãžã§ã³ãæã€ããšã«ãªããBazel ã¯ãone-version (ã¢ãããŒãžã§ã³ãšã) ãªãã«ãã»ããŒã«ãšããŠç¥ãããŠããã®ã§ãåé¡ç©ºéãã©ããããã®ãã®ãªã®ãã«é¢å¿ããã£ãã
Python ã䜿ã£ãŠåæ£ã¹ã³ã¢ãé©åœã«äœãã忣ã¹ã³ã¢ãæå°åããè»žãæ¢ããŠããå埩ã¢ã«ãŽãªãºã ãæžãããå°äžéã®è·¯ç·å³ã¿ãããªçµµãåºæ¥ããã£ããå¶çºçã«æ°å以äžã®ãã«ãããŒã¹ããã£ãããã¡ãžã£ãŒããŒãžã§ã³ã§ã¯ã©ã¹ã¿ãŒåãããšæ°åãããã«åãŸãã¿ããã ã£ãã
ãã®éãç¥ãåãã®ãšã³ãžãã¢ã®äººãã¡ã¯ Slack ã® DM ã«å ¥ã£ãŠããŠãåãã©ãããŠãããã§ãã¯ããŠãããããäœæ¥äžã®é¢çœããã®ãå ±æããŠãããããä»äºã®èŠé ãæããŠããããããããã®äžã§ã 2åæãåºãŠã人ãã¡ã¯ç€Ÿå Scala ããŒã ã® Shane Delmoreãããš Ãlafur Geirssonããã ãåãèªåã®ã©ãããããã䜿ã£ãŠé ã ãšæ°åã®ã¿ãŒã²ããã«å¯Ÿã㊠Pants ã³ãã³ããå®è¡ããŠãã®ãèŠãŠãShane 㯠scoot ãšãã Mesos äžã§æ°åã®ãã«ããåæå®è¡ã§ãã CI ã€ã³ãã©ã§å®è¡ããŠãããã
Olaf ã¯ããã®åŸ bazel-multiversion ãšãªããã®ã®ãããã¿ã€ããæžããŠãããåœæãã©ã®ã¬ã€ã€ãŒã§ Coursier ã®äŸåæ§è§£æ±ºãå®è¡ããã¹ãããšããè°è«ãå
éšã§ãã£ãŠãå€éšã©ã€ãã©ãªã衚ã jar_library
ãš Pants ã sbt ã®ããã«æ«ç«¯ã¿ãŒã²ããã§ãã scala_binary
ã¬ãã«ã§èµ°ãããã¹ããšãã 2掟ããã£ãã
11æé ã«ã¯ãåã 3rdparty/jvm (JVMäžã®å€éšã©ã€ãã©ãªã®ãµããŒã) ã®ãã©ã€ããŒãšãªãã瀟å
ã§ã® 3rdparty/jvm ããŒãããããæžãããåã®äºãç¥ããªã人ããããšæãã®ã§èªå·±ç޹ä»ãããšãå㯠sbt ãšãã Scala ã³ãã¥ããã£ãŒã§äž»ã«äœ¿ããããã«ãã»ããŒã«ã®ãªãŒãããã°ãããã£ãŠããŠãsbt ã«ãã㊠cached resolutionãeviction warningãversionScheme
ãªã©é¢é£ããæ©èœã®èšèšãšå®è£
ãè¡ã£ãŠããããã®ãããªçµéšãšãã«ãããŒã¹ã®èª¿æ»ã§åŸãããããŒã¿åºã¥ããŠãããŒãžã§ã³çªå·ã Semantic Version ã§ãã±ããåããããšããå§ããã
PythonåŽã§ã¯ãHenry Fullerããã 3rdparty/python ã®ãã€ã°ã¬ãŒã·ã§ã³ãš Bazel äžã® Python ãµããŒãæŠç¥ãæ åœããŠããã
å人ã§ã®ãµã€ããããžã§ã¯ããšããŠã¯ã10æã«å€ãã®ã¹ã¿ããã®ä»²éãšäžç·ã« ScalaMatsuri 2020 ã®ãªãŒã¬ãã€ãºã«ãæäŒããããåãæ åœããã»ãã·ã§ã³ã® 1ã€ãšããŠä»®æ³åããããã«ãœã³ãè¡ããåå è ã Scala ã³ã³ãã€ã©ã sbt ãªã©ã« pull request ãéããæäŒããããã
11æã¯ãWeehawken-Lang1 ãšããããã»ãã£ãªãã£ãŒã€ãã³ããéå¬ã㊠Scala ã«ãããç䟡æ§ã®è©±ãããã
2021
2021幎ã®2æã«ãScala Center é¡§åäŒè°ã® Twitter代衚ã«å°±ä»»ããã
å¶ç¶ã ããäžåºŠãã®åæ¥ã« JFrog瀟ã Bintray ã®ãµãŒãã¹çµäºãã¢ããŠã³ã¹ãããScala CenterãVirtusLab瀟ãLightbend瀟ãšãå ±åã§ãsbt ã®ãã©ã°ã€ã³ã»ãšã³ã·ã¹ãã ã®å®å šãªç®¡çãšãã«ãæå³è«ã®æç¶ãç®çãšããŠã¿ã¹ã¯ãã©ãŒã¹ãçµãã ã幞ã JFrog瀟ã open source ã¹ãã³ãµãŒã·ãããæäŸããŠãããã®ã§ sbt ã®ãã©ã°ã€ã³çŸ€ãã€ã³ã¹ããŒã©ãŒã¯ Scala Center ã«ã©ã€ã»ã³ã¹ãããã¯ã©ãŠãã»ãã¹ã㪠Artifactory ã®ã€ã³ã¹ã¿ã³ã¹ã«åãç§»ãããšãã§ããããã®ãããªèšæã®ã»ãã¥ãªãã£ã»ããããé€ããšãåã Twitter瀟ã«åšç±ããŠããéã¯åºæ¬çã« sbt ã®äœæ¥ã¯é±æ«ã«ã ãè¡ã£ãŠããã
3æã«ã¯ Angela Guardiaããã Build/Bazel Migration ããŒã ã«å ¥ç€ŸããŠãMartin Duhemãããšå ±ã« 3rdparty/jvm ã¯ãŒã¯ã¹ããªãŒã ã«åå ãããããŒã¿ã«åºã¥ããæ å ±ã䜿ã£ãŠ 3rdparty/jvm ã®ã°ã©ãã調æŽããããšããã¢ã€ãã£ã¢ãåãã¡ã«ã¯ãã£ãã®ã§ãAngela 㯠bazel-multiversion ã® YAML åºå ãå®è£ ããŠãæ¯æ©èµ°ã Jenkins ãžã§ãã§ JARãã¡ã€ã«è¡çªæ€ç¥çšã®ãªã³ã¿ãŒãå®è¡ããŠããã°éèšããããšããããšãå®çŸãããè¡çªæ€ç¥ããªã³ã¿ãŒã§è¡ããšããããšã¯ Bazelcon ã§åãçºè¡šãã Resolving Twitter’s 3rdparty/jvm with bazel-multiversion ã§ã玹ä»ããã
6æã«ã¯ãrules_scala ã® collect_jars
ãã§ãŒãºãã«ã¹ã¿ã åããŠæ«ç«¯ã¿ãŒã²ããã¬ãã«ã§èªåçã«è¡çªã解決ããå®è£
ãè¡ã£ããåã®ããŒã¯ã§ã¯ããã¯ãtertiary resolutionã(3次解決) ãšåŒãã ïŒã¡ãªã¿ã« Bazelcon 2022 ã«ãããŠãããããã«çºå±ãããåãããŒã¯ã Airbnb瀟ã«ãã£ãŠçºè¡šãããïŒã
倧ãç®ã®ããŒã ãšé·ã todo ãªã¹ããšããç¶æ³ã§ãããŒã ãäœãããã¹ããã®æé©ååé¡ã®ããšãèããããã«ãªã£ãŠããããã°ããã¯ãããã10æåæé ãªã¹ãããšãã JIRA ãã±ãããã©ã³ã¯ä»ããã Google Spreadsheet ã管çããŠããåã¯ããã®ãããªããŒã¿ã»ããªãã³ãªææ³ã todo ãªã¹ãã«é©çšããããšããã¡ã¿ã¢ããªã·ã¹ããšåæã«åŒãã§ãããåãéžãã ã¡ããªãã¯ã¯ãã¿ã¹ã¯ã黿¢ããŠãã鿥çäŸåã¿ãŒã²ããã®æ°ã ãå€ã®éãUC Berkeley ããã®ã€ã³ã¿ãŒã³ Hannah Ku ãããš Angela ãååããŠãããã X æåæé ãªã¹ãããã¡ãããšããèªåæŽæ°ãããããã·ã¥ããŒãã«ä»äžãããããã§èª°ããªã³ããŒãããŠããæãã€ã³ãã¯ãã®å€§ããã¿ã¹ã¯ãéžãã§ãããããšãã§ããããã«ãªã£ãã
Bazeläºæã®ã¿ãŒã²ããæ°ãå¢ããã«ã€ããbuildcache ã®ã¹ã±ãŒã©ããªãã£åé¡ã«è²ã ã¯ãŸãå§ããŠããã確ããã®ææã« Ekaterina (Katya) Tyurinaããã buildcache ã®ã¹ã±ãŒã©ããªãã£ã®å£ã«ã€ããŠè©³çްãªåæã¬ããŒããæžããŠãTCPãããã¡ãŒã®å²ãåœãŠããhermecity (ããã«ãå¯éæ§ã)ãå£ããŠããå¯èœæ§ãææããã
å°ãæ¹åæ§ãå€ããŠã第3ååæã«ã¯ããŒã¿åŠçãžã§ãã®ããã® Bazel ãµããŒãã®èšèšãšå®è£
ãè¡ããScalding ã®å€ããšãã workstream ãç«ã¡äžãããããã«ãã£ãŠããããã€ã»ãã€ãã©ã€ã³ãããŒã¿ã»ãã©ãããã©ãŒã ãæã€ç°ãªãããŒã ãšè©±ãæ©äŒãã§ãããããã§åãæ°ã¥ããŠããŸã£ãã®ã¯ã瀟å
ã®ä»ã®äººãã¡ã«å¯ŸããŠåãã¡ã補åãšããŠæäŸããŠããã®ã¯ Pants ã§ã Bazel ã§ãç¡ãããã«ããã§ãã£ãŠãã¹ã ãŒãºã«ãã€ã°ã¬ãŒã·ã§ã³ãè¡ãã«ã¯ drop-in ãªãã¬ãŒã¹ (ãå·®ã蟌ãã ãã§è¯ãäºæè£œåã) ãå¿
èŠãšããããšã ã£ãããã®ãããå㯠bazel
ã·ã§ã«ã»ã¹ã¯ãªããã䜿ã£ã bazel bundle
ãšãããPantsäºæã®ãããã€ã»ã€ã¡ãŒãžãçæããæ¡åŒµã³ãã³ããå®è£
ããŠãããã䜿ã£ãŠ Scalding ã® Bazel ãµããŒããå®è£
ããã
Flock ã®äžã§ã¯ãããŒã éã§çµæ§æµåçã«ãšã³ãžãã¢ã®ç°åããã£ããæ§ã ãªããŒã ãæž¡ãæ©ããŠããã¹ã±ããŒä¹ãã®ã¹ã¿ããã»ãšã³ãžã㢠Adam Singerããã Scala ããŒã ã«å ¥ã£ãŠããŠãããã« Bazel Migration ããŒã ã®éèŠãªã¯ãŒã¯ã¹ããªãŒã ã®ãªãŒã圹ãåããããã«ãªã£ãã圌ã¯èªå® ã«å€§ãããã¹ã¯ããããã·ã³ãæã£ãŠããŠå šãŠã® JUnit ã¿ãŒã²ãããé£ç¶çã«èµ°ãããŠã©ãã§ Bazel ã®äºææ§ãå£ããããæ¢ããšããããšããã£ãŠãããããã«ããããã¡ã€ã©ãŒã»ããŒã«åšãã«æ £ããŠããŠãè²ã ãªåé¡ãæ€ç¥ããŠãããäŸãã°ãaction cache ããã©ãããã©ãŒã äŸåã§ããããã« Mac ã®ã©ãããããäžãããªã¢ãŒãã»ãã£ãã·ã¥ãããŸãå©çšã§ããªããšããåé¡ãææããã®ã¯åœŒã ã£ããšæããããã¯ãçŸåšãç¶ããŠãã Bazel ã®èª²é¡ã® 1ã€ã ããããã®æéã§äžçªé¢çœã話ãããŠããã®ã Adam ã ã£ãã
ãŸããBazel ã®å šç€Ÿæ¡çšãžã®æºåäœå¶ãšããŠå éšããã¥ã¡ã³ããŒã·ã§ã³ã»ãµã€ããgo/bazelãã®ç«ã¡äžãããªãŒãããŠããã¥ãŒããªã¢ã«ããã©ãã«å¯Ÿçã¬ã€ããªã©ããããããŸã瀟å ç ä¿®çšã« Pants ãš Bazel ã®éãã解説ãããBazel at Twitterãè¬åº§ãäœæããã
12æã®äžé£ã® log4j è匱æ§ä¿®æ£ã«è¿œéããŠãå éšã®äŸåæ§ã®ããããè¡ã£ãããèšæã« sbt 1.5.x ã·ãªãŒãºã®ããããè¡ã£ãããããã
ãŸãã2021幎ã«ã¯ãEugene 㯠1幎åã®æ¡çšæã«æããã«ééã£ãã¬ãã«ä»ãããããã颿¥ããã»ã¹ã«ãããŠæ¥çå ã«ãããçµæŽã¯ååã«çè§£ãããããè©äŸ¡ãããŠããªãã£ãã(以äžç¥)ããšããçç±ã§ã¹ã¿ããã»ãšã³ãžãã¢ã«ææ ŒãããŠããã£ã (David RahnãããããããšãããããŸã!)ããã® Staff Engineer ãšããã®ã¯ãã·ãªã³ã³ãã¬ãŒäŒæ¥ã§ã®åœ¹è·åã®1ã€ã§ãã·ãã¢ãšã³ãžãã¢ã® 1段äžã ãæã¯ç®¡çè·ã«ãªãããææ Œã®æ¹æ³ãç¡ãã£ãããäžå¹Žã§ãæè¡çã§ãã£ãªã¢ãç©ã¿ãã人åãã«åç€Ÿã«æ®µéå¶ãäœããããé°å²æ°ãšããŠã¯ãããŒã ã®æ çµã¿ãè¶ ããã·ã¹ãã èšèšãè¡ãã®ã Staff Engineer ã§ãéšçœ²ã«æ°åå± ãŠå®éšçãªäºãè¡ã£ãããã Principal Engineer ãšãã人ãã¡ãããã«ãã®äžã®ã©ã³ã¯ãšããŠããã
2022
Engineering Effectiveness 課ãšãã以å€ã® Products ã Revenue ãšãã£ãéšçœ²ã®é¢ä¿ãå°ã解説ããããåã㊠Yi ãšã Ity ãã瀟å ã®ä»ã®ããŒã ã®äºããã«ã¹ã¿ããŒããšèšã£ãŠããã®ãèãããšãã¯éåæããã£ãããåŸã«ãªã£ãŠãããã®ããŒã ã¯ã¢ããªããšããŠã®è£œåãæäŸãã Build ããŒã ããéã£ãããšããããšã«çŽåŸããããããã®ããŒã ã¯èªç±ã«ä»ã®ãã«ãã»ã·ã¹ãã ãæ¡çšããããšãã§ããããã ïŒã¢ãã€ã«ã»ã¢ããªã¯å®éããããŠããïŒããã®ãã¡ãåãä»ã®ããŒã ã EE ã®ã«ã¹ã¿ããŒã§ãããšããæèã«æ £ããŠãã£ãã
Build ããŒã ã¯ãäžé±éç¶ããªã³ã³ãŒã«ã®åœçªããã£ããäž»ãªè²¬åã¯éåžžã®æ¥åæéå ã« JIRA çµç±ã§ã«ã¹ã¿ããŒã»ãµããŒããæäŸããããšã ãåœåã¯ãå€ãã®ãã±ããã Pants ã®è³ªåã§ãã«ã¹ã¿ããŒã®ã¢ã·ã¹ããããã®ã«ä»ã®ããŒã ã¡ã€ãã®å©ããå¿ èŠãšããããPants ã®ç¥èãåŸãããšã«æ £ããŠããŠã質åã Bazel ãã€ã°ã¬ãŒã·ã§ã³é¢é£ã«ãªã£ãŠãããšãBuild ããŒã ãšããŠè¯ãã«ã¹ã¿ããŒã»ãµããŒããæäŸã§ããããšã楜ãããªã£ãŠããã
ãªã³ã³ãŒã«ã®è²¬åã«ã¯ Source ã¢ããªãã®é害察å¿ãå«ãŸãããã¢ã¡ãªã«ã®ã¿ã€ã ãŸãŒã³ã«äœãã§ãäºæ ããã£ãŠãåãš Yi ã«ããåã£ãŠããŠãCI ãã€ãã©ã€ã³ã buildcache ã®é害ããã£ãå Žåã«ã¯ã¡ã¢ãªäœ¿çšã Maven XML ãã¡ã€ã«ã®ãã£ãã·ã¥çããããã°ããŠãåŸæ¥ã«ã¯äºåŸæ€èšå ±åæžãæžããã
ã€ã³ãã©ã»ã¬ãã«ã§ã®èª²é¡ã® 1ã€ãšã㊠IDEã®ãµããŒããæ®ã£ãŠããã2022幎ã®2æã«ã¯ãIntelliJ IDEA ã€ã³ããŒãã«ãããã ã鿥çäŸåæ§ãåæ ãããªããªã©ã®å éšã³ãŒãã®åé¡ãããã€ã調æ»ããã
2æã«ã¯æ«ç«¯ã¿ãŒã²ããã«ããã exclude æ©èœããŸãããŠã collect_jars
ãã§ãŒãºã䜿ã£ãŠå®è£
ããŠãäœ¿çšæ¹æ³ã®ããã¥ã¡ã³ããŒã·ã§ã³ãæžããã
3æã®ç€Ÿå Hack Week ã§ã¯ãBazel Hack Week ãäŒç»ããŠãBazel ã䜿ã£ãŠããã£ãããæ®ã£ãŠãã Pants æ©èœãå®è£ ããŠããã£ãããããHack Week äžã« shading æ©èœã Node.JS ãµããŒãããããã¿ã€ããããBazeläºæ % ãåäžããããã®èŸºããããäŒè©±ãå®éã«ãããã€ãããã€ã°ã¬ãŒã·ã§ã³% ã«ç§»ã£ãŠãã£ããšæãã
æ°åã®ã¡ã³ããŒãæããããBazel Migration ã¯ãããžã§ã¯ããšããŠãŸã ä¹ã£ãŠããã®ã§ãçµéšå€ãé«ãã®ã¡ã³ããŒãããŒã ã«åå ããŠããã
- ã¡ãã£ã¢ããŒã ããã¯ãDiego Puppinãããç§»ç±ããŠããŠãNode.JS ãã®ä»ã®ã¯ãŒã¯ã¹ããªãŒã ãåŒã£åŒµã£ãŠãã£ãã
- Meta瀟ãã Adam Hani Schakakiãããå ¥ã£ãŠããŠãã¯ãŒã¯ãããŒã®ãã€ã°ã¬ãŒã·ã§ã³ã«åå ããããTalha Pathanãããšããã€ã³ã¿ãŒã³çã®æå°ãè¡ã£ãŠãå€ã®éã« Golang ã®ãµããŒããå®è£ ããŠããŸã£ããAdam 㯠Python ã®ã³ãŒãã®ã¬ãã¥ãŒã«è©³ããã³ã¡ã³ãããã·ãã·æžããŠãããŠãéåžžã«å匷ã«ãªã£ãã
- JetBrains瀟ãã Liana Bakradzeãããå ¥ç€Ÿãããæåã¯ãã€ã°ã¬ãŒã·ã§ã³ã®ã¡ããªãã¯è¿œè·¡ã®æ¹åãè¡ã£ãŠãåŸã»ã©ã¯äºéãã«ããæžããããã« Bazel ã®ã¿ã®ãã«ããè¡ãããšãç®çãšãã Pants 廿¢ã¯ãŒã¯ã¹ããªãŒã ããªãŒãããã
5æã«ã¯ scala_library
rule ã®ã¿ãŒã²ããããšã® strcit-deps æ©èœãå®è£
ãããããã«éãã Pants ã¯ã¿ãŒã²ããããšã«èšå®ãåãæ¿ããããã®ã«ãéåžžã® Bazel ã ãšãããã§ããªããšããå Žé¢ãçµæ§ãã£ããåãå®è£
ãããã1ã€äŸã ãšãProtobuf ã®çæãããã®ã«ã¿ãŒã²ããããšã« protoc_version
ãåãæ¿ãããšãããã®ããã£ããããã¯ãHadoop ã protobuf-java 2.x ç³»ãå¿
èŠãšããã®ã§ãScalding ãžã§ãã®ç§»è¡ã®ããã®å€§ããªãã¬ãŒã¯ã¹ã«ãŒãšãªã£ãã
ãŸããScalaãThriftãJava ã®ã¿ãŒã²ããã®ããã®ãããªãã·ã¥æ©èœãå®è£ ããããã㯠rules_jvm_export ãšããŠãªãŒãã³ãœãŒã¹åãããrules_jvm_export ã¯ãã¿ãŒã²ããã®äŸåæ§ã«åãããŠæ£ãã POM ãã¡ã€ã«ãäœãããšãç®æããŠããã
ãã 1ã€åãé¢ãã£ãŠããã®ã¯ã¿ãŒã²ããã»ã¬ãã«ã§ã®ãããã»ã³ãŒãåæã ãæªäœ¿çšã® JAR ãã¡ã€ã«ãæ€ç¥ããããã« JVM ã®åŒã³åºãã°ã©ãã䜿ã£ã classpath-verifier#10 ãå®è£ ãããŸããScalac ãã©ã°ã€ã³ã§ãã ClassPathShrinker ã®ç§»æ€ãè¡ã£ãããããã»ã³ãŒãåæã®ç®çã¯å€§èŠæš¡ã¿ãŒã²ããã®ãã«ãæéã®åæžãç®çãšããŠããã
11æãšãªããšã倧éšåã®ã©ããããã䜿çšãšãããã€ã®å€ã㯠Bazel ãžãšç§»è¡ããŠãããããããããã€ãã®å¹çåã®èª²é¡ã¯æ®ã£ãŠããŠãæåŸã®æ°æ¥ã§ãåãã¡ã¯å¹çæ¹åãè¡ã£ãŠãããåã®æåŸã® phab (瀟å ã§ã® pull request ã®åå)ã¯ãã³ãŒãããªãŒãºãè§£ããåŸã§ãCoursier CVE察çãè¡ã£ã bazel-multiversion ãžãšæŽæ°ãããã®ã ã£ãã
æ¯ãè¿ã£ãŠèŠããšãèªåã®ãã£ãªã¢ã®äžã§ãæãå®ãã®å€ã 2幎éã ã£ãæ°ãããããã®å°ç€ãšãªã£ãã®ã¯ãããšãã®æ¥µã¿ããšãèšããã Twitter瀟ã®ã«ã«ãã£ãŒã«ãã£ãŠå¹ãããã¯ãŒã¯/ã©ã€ãã»ãã©ã³ã¹ãšãEngineering Effectiveness ã®ãªãŒããŒé£ã§ãã Nagesh NayuduããããBuild/Bazel Migration ããŒã ã®ãããŒãžã£ãŒ David Rahnãããš Ioana Balasãããå¿ççå®å šãæ ä¿ããŠãããããšã«å°œãããåãããããããšãæ£ã«è·å Žãåã«ãã£ãŠæ¬²ããããšã§ãããšããæå³ã«ãããŠãå㯠Engineering Effectiveness ã®ããã·ã§ã³ã«ããžãã¹ã»ã¢ã©ã€ã³ã¡ã³ããèŠã€ããããšãã§ãããåã¯ãNagesh ã David ãšã® 1-on-1 ããã€ã楜ãã¿ã«ããŠãŠãèªåã®ã¢ã€ãã£ã¢ãå ±æãããã圌ãã®é«å°ããã®èŠç¹ããåŠãã ãããããŸããDavid ãä»ã®ã·ãã¢ã¡ã³ããŒãä¿¡é Œã§ããããé£é¡ã§ãé£ã³èŸŒãã§ããããšãã§ããããã® OneTeam ãšããç±æ°ãåã¯å¿ããããšã¯ãªããšæãã
Twitter瀟ã«ããã Scala
å ¬éãããŠããªãããšã¯æžããªãããTwitter瀟㯠Scala ã«é¢ããŠããªããã©ã³ã¹ã®è¯ãã¹ã¿ã³ã¹ãåã£ãŠããŠãScala èšèªã®è±å¯ãªæ©èœãå šãŠæšãŠãããã¿ãŒ Javaãã§ãç¡ããã°ãè¡ãéãã§ãWorse Haskellã(ãããã¯ææ ã蟌ã㊠Type Astronaut ãåã®å®å®é£è¡å£«ã)ã§ãç¡ããå€ãã®ãšã³ãžãã¢ããããšããçç±ããã£ãŠãTwitter瀟ã®ã³ãŒãããŒã¹ã¯ãã®äžåºžããã£ãŠãã:
- äžå€æ§ (immutability) ãæšå¥š
- Scala ã®ã³ã¬ã¯ã·ã§ã³ã»ã©ã€ãã©ãªã䜿ã
- æé»ã®å€æã¯é¿ãã
com.twitter.util.Future
ãçšããŠãäžŠè¡æ§ãåã§è¡šã
ãã®å²åŠã¯ Scala SchoolãEffective ScalaãScalding ããã¥ã¡ã³ããŒã·ã§ã³ãScala at Scale at Twitter (2016) ãªã©ãããåãããšæããTwitter瀟ã¯ããŒã«ã«åãããã¹ã±ãžã¥ãŒã©ãæã¡ããã£ã³ã»ã«å¯èœãªç¬èª Future å®è£ ã Scala 2.10 ã fork/join ã䜿ã£ããã®ã 2013幎ã«ãªãªãŒã¹ãããã£ãšåããæã£ãŠããäºã«æ³šæã
Scala ã®ããšãããŸãç¥ããªã人åãã«å°ã説æãããšãScala ã®æšæºã©ã€ãã©ãªã¯ã³ã¬ã¯ã·ã§ã³ã»ã©ã€ãã©ãªãå«ã¿ãJVM ã«ãããåºæ¬çãªããŒã¿æ§é ã§ããé
åããªã¹ããMap
(ãã£ã¯ã·ã§ããª)ã®ãšã¬ã¬ã³ããªå€æãå¯èœãšããJava ã®ãããªåœä»€åœ¢èšèªã ãšæ°è¡ã®å®åœ¢ãèŠããã³ãŒããç°¡æœã«æžãããšãã§ããã
äž»èŠãšãªã£ãä»ã®èšèªã®ã©ãããã Scala ã¯ã©ã€ãã©ãªäœè ã«èšèªãã®ãã®ãæ¡åŒµããŠç¬èªã®æ¹èšãå®çŸ©ããã (ããã¯ãã¡ã€ã³ç¹åèšèª DSL ãšåŒã°ãã)ãæ¢åã® Java ã©ã€ãã©ãªãå ã¿èŸŒãã§ããè¯ãããããããŒã»ãšã¯ã¹ããªãšã³ã¹ãæäŸããããšãå¯èœãšãããScalding ã¯ãã®æããäŸã§ãJava ã ãš1ããŒãžååãŸããããªã決ãŸãã³ãŒããæžããªããã°ãªããªããã㪠Hadoop ãžã§ãã Scala ã ãšæ°è¡ã§æžããããã«ãªãããã®èãæ¹ã¯ Spotify瀟ã®ãããªä»ç€Ÿã«ãåãç¶ãããŠã圌ã㯠Dataflow äžã« Scio ãäœæããŠãããããã
Twitter瀟ã¯ããã§ã¢ãã€å€æ§æ§ã®ããè·å Žãäœãããšã«åªåããŠããŠãæ§å¥ã人皮ãšãã£ã軞ãããã ããæ§ã ãªã¹ãã«ã®èæ¯ãæã£ãã²ãšãéãŸã£ãŠããŠãããå士å·ãæã£ããšãã¹ããŒããããã°ãããã°ã©ããšããŠãã£ãªã¢ãå§ããã°ã£ããã®äººãã¡ããããå¥ã®èšãæ¹ããããšãæ°åã®ãšã³ãžãã¢ããããšããããæ°çŸä»¥äžã®å¥³æ§ã Scala ã®ã³ãŒããæžããŠããèšç®ãšãªãã圌女ããã©ãããžè¡ã£ãŠããŸããšãããšãããã ã倿§æ§ã倱ãããŠããšãæããŸããã
Scala Center ãžã®åé
Scala Center ããµããŒããããã
çµæžç¶æ³ãäžåãã«ãªããšäŒæ¥ç³»ã®ã¹ãã³ãµãŒã¯åºè³ãæžãããã«ãªããããããªãããå人ããã°ã©ããã«ã³ãããŠScala Center ããµããŒãããããšãã§ãããçŸåšã®ã¢ã¡ãªã«ã¯ã€ã³ãã¬ã 8% ãããæ°ãªãŒããŒãæ°åããçŸéåããŠãããæã¡æ ªã®äŸ¡å€ãã©ãã©ãäžãã£ãŠããã ããã
åãã¡ã¯æ°å人ããããã ãããå šå¡ã RSU ãéè·éã® 3% ã§ãåããããšãã§ããã°ãScala Center ãšããŠã¯çžåœã®é¡ãéãŸãã¯ãã ãå Tweep ãšããŠã¯ãScala ãšã³ã·ã¹ãã ãšãScala ã®ä»äºçžå Žãç¶ç¶ããŠãããããšã«å©å·±çãªäŸ¡å€ãããããã ããCVE ã®ä¿®æ£ãè¡ã£ããã¯ãŒã¯ã·ã§ãããã«ã³ãã¡ã¬ã³ã¹ã®éå¶ãè¡ã Scala Center ã«å¯ŸããŠä»åŸ Twitter瀟 ããµããŒããç¶ãããã¯äžæãªæãã ã
次ã®ã¹ããã
åã¯ã¢ããªãã Bazel ãžãšç§»è¡ãããããã« Twitter瀟ã«å ¥ç€ŸããŠãå€ãã®äººãšååããŠããã®ç®çã¯éæããããšãã§ããä»åãã¡ã¯åºãŠããæã ãæ°ãªãŒããŒã¯åãã¡ã®ãã°ããã®å¬äŒã¿ã®åºè³ããŠããããšèããããšãã§ããããŸã蟣油ãäœã£ãããããã°ãæžãããã§ããã°è¯ãããªãšæã£ãŠããã
ãã®åŸã§ã次ã®ããã°ãããã¬ã ã解決ããããã®æ°ããããŒã æ¢ããå§ãããšæãããããããå®ãããé¡ãããŸãã